CN112073640B - Panoramic information acquisition pose acquisition method, device and system - Google Patents
Panoramic information acquisition pose acquisition method, device and system Download PDFInfo
- Publication number
- CN112073640B CN112073640B CN202010966678.0A CN202010966678A CN112073640B CN 112073640 B CN112073640 B CN 112073640B CN 202010966678 A CN202010966678 A CN 202010966678A CN 112073640 B CN112073640 B CN 112073640B
- Authority
- CN
- China
- Prior art keywords
- face recognition
- image acquisition
- face
- equipment
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N25/41—Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/70—Multimodal biometrics, e.g. combining information from different biometric modalities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/725—Cordless telephones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a panoramic information acquisition pose acquisition method, a panoramic information acquisition device and a panoramic information acquisition system. And after the face recognition equipment acquires the self pose parameters, acquiring the pose parameters of the image acquisition equipment according to the relative included angle. The pose parameters of the image acquisition equipment are acquired according to the pose parameters of the face recognition equipment in combination with the relative included angles, so that the problem of angle deviation caused by the fact that panoramic images acquired by the image acquisition equipment are spliced due to the fact that no pose parameters exist is solved.
Description
Technical Field
The invention relates to the technical field of information acquisition, in particular to a panoramic information acquisition pose acquisition method, a panoramic information acquisition pose acquisition device and a panoramic information acquisition pose acquisition system.
Background
In the data acquisition stage of the indoor three-dimensional model, different shooting point data (including depth data and RGB image data) are collected by using special equipment. And after the data collection is finished, obtaining an RGBD image through the calibrated camera parameters, the RGB image and the depth image. RGB is a color standard in the industry, and various colors are obtained by changing three color channels of red (R), green (G), and blue (B) and superimposing the three color channels on each other, where RGB represents colors of the three channels of red, green, and blue, and the standard includes almost all colors that can be perceived by human vision, and is one of the most widely used color systems at present. A Depth Map (Depth Map) is an image or image channel containing information about the distance of the surface of a scene object from a viewpoint. Where the Depth Map is similar to a grayscale image except that each pixel value thereof is the actual distance of the sensor from the object. Usually, the RGB image and the Depth image are registered, so that there is a one-to-one correspondence between the pixel points.
In the prior art, an RGBD (red green blue) graph can be converted into point cloud of a single camera point by combining camera poses. This requires finding out the coordinates of the different shot points in the global coordinate system, so that the point cloud data can be spliced into a point cloud model of the complete house.
In the prior art, AR (virtual reality) technology is generally used for real-time positioning and panoramic information acquisition. Although the AR engine can return the pose information of the mobile phone at any time, the pose of the mobile phone cannot be associated with the shooting angle of the panoramic camera, so that the overall angle deviation of the result is caused.
Disclosure of Invention
The embodiment of the invention aims to solve the technical problem that: the method, the device and the system for acquiring the panoramic information acquisition pose solve the problem that the overall angle deviation of the result is caused because the mobile phone pose cannot be associated with the shooting angle of a panoramic camera in the prior art.
According to an aspect of the present invention, there is provided a panoramic information collection pose acquisition method, including:
matching face information acquired by face recognition equipment in a panoramic image acquired by image acquisition equipment; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions;
obtaining a relative included angle between the face recognition equipment and the image acquisition equipment according to the position of the face information in the panoramic image;
and acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle.
Optionally, the fixing of the relative position setting of the face recognition device and the image acquisition device includes:
the relative position, the relative distance and the relative angle of the face recognition equipment and the image acquisition equipment are all fixedly arranged.
Optionally, the method further comprises:
and when the face recognition equipment acquires the face information, the image acquisition equipment acquires a panoramic image.
Optionally, the method further comprises:
the face recognition apparatus acquires any one of face information as the face information.
Optionally, the method further comprises:
after the face recognition device acquires the face information, when the definition and the angle of the face information are determined to accord with preset threshold values, the face recognition device informs the image acquisition device to acquire a panoramic image;
the preset threshold value is set according to definition and an angle range required by clear facial information identification.
Optionally, the obtaining a relative angle between the face recognition device and the image acquisition device according to the position of the face information in the panoramic image includes:
comparing the face information in the panoramic image;
determining the relative position of the face information and an image acquisition device according to the position of the face information in the panoramic image;
and calculating to obtain a relative included angle between the face recognition equipment and the image acquisition equipment according to the relative position between the face information and the image acquisition equipment.
Optionally, the acquiring the pose parameter of the image capturing device according to the pose parameter of the face recognition device and the relative included angle includes:
acquiring pose parameters of the face recognition equipment;
and obtaining the pose parameters of the image acquisition equipment by combining the relative included angle adjustment according to the pose parameters of the face recognition equipment.
According to another aspect of the present invention, there is provided a panoramic information collection pose acquisition apparatus including:
a matching unit for matching the face information acquired by the face recognition device in the panoramic image acquired by the image acquisition device; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions;
the relative angle acquisition unit is used for acquiring a relative included angle between the face recognition device and the image acquisition device according to the position of the face information in the panoramic image;
and the pose parameter acquisition unit is used for acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle.
Optionally, the apparatus further comprises:
and the face recognition unit is used for acquiring any face information as the face information and sending the matching unit to match in the panoramic image acquired by the image acquisition equipment.
Optionally, the face recognition unit is further configured to, after the face information is obtained, notify the image capture device to capture a panoramic image when it is determined that the sharpness and the angle of the face information meet a preset threshold; the preset threshold value is set according to definition and an angle range required by clear facial information identification.
Optionally, the apparatus further comprises:
and the panoramic image acquisition unit is used for acquiring panoramic images.
Optionally, the relative angle obtaining unit is specifically configured to:
comparing the face information in the panoramic image; determining the relative position of the face information and an image acquisition device according to the position of the face information in the panoramic image; and calculating to obtain a relative included angle between the face recognition equipment and the image acquisition equipment according to the relative position between the face information and the image acquisition equipment.
Optionally, the pose parameter acquiring unit is specifically configured to:
acquiring pose parameters of the face recognition equipment; and obtaining the pose parameters of the image acquisition equipment by combining the relative included angle adjustment according to the pose parameters of the face recognition equipment.
According to another aspect of the present invention, there is provided a panoramic information collection pose acquisition system including a face recognition apparatus and an image collection apparatus that are fixed in relative position settings, wherein:
the face recognition device is used for acquiring face information;
the image acquisition equipment is used for acquiring a panoramic image;
the control equipment is used for matching the face information acquired by the face recognition equipment in the panoramic image acquired by the image acquisition equipment; obtaining a relative included angle between the face recognition equipment and the image acquisition equipment according to the position of the face information in the panoramic image; and acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle.
Optionally, the face recognition device is further configured to notify the image capturing device to perform panoramic image capturing when the face information is acquired.
According to another aspect of the present invention, there is provided a computer-readable storage medium storing a computer program for executing the method described above.
According to another aspect of the present invention, there is provided an electronic apparatus including:
a processor;
a memory for storing the processor-executable instructions;
the processor is used for reading the executable instructions from the memory and executing the instructions to realize the method.
Based on the scheme provided by the embodiment of the invention, the method mainly comprises the following beneficial effects:
matching face information acquired by face recognition equipment in a panoramic image acquired by image acquisition equipment; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions; obtaining a relative included angle between the face recognition equipment and the image acquisition equipment according to the position of the face information in the panoramic image; and acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle. According to the scheme of the embodiment of the invention, the interaction between the face recognition equipment and the image acquisition equipment is utilized, the pose parameters of the image acquisition equipment are acquired according to the pose parameters of the face recognition equipment and the relative included angle, and the problem of angle deviation caused by the fact that panoramic images acquired by the image acquisition equipment are spliced due to the fact that no pose parameters exist is solved.
And after the face recognition equipment acquires the self pose parameters, acquiring the pose parameters of the image acquisition equipment according to the relative included angle. Panoramic images collected by the image collecting equipment at all positions of a room can be automatically spliced into three-dimensional images conveniently, a vectorized extremely-simple three-dimensional model of the room can be deduced by collecting the panoramic images of the point positions of the room through simple image collecting equipment, and the requirement of quick and simple whole-house reconstruction can be met.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The invention will be more clearly understood from the following detailed description, taken with reference to the accompanying drawings, in which:
fig. 1 is a schematic flow chart of a panoramic information acquisition pose acquisition method according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a panoramic information acquisition pose acquisition apparatus according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a panoramic information collection pose acquisition system according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
In the embodiment of the invention, different panoramic images can be acquired at different positions in a room through the image acquisition equipment, and then the point clouds of the different panoramic images are spliced into the whole house model. But the image splicing is difficult due to the uncertain position and angle of the image acquisition equipment. Thus, it is necessary to first determine the pose parameters of the image capturing apparatus. However, the conventional image capturing device does not have the function of acquiring the pose parameters, and therefore, other technologies are required to acquire the pose parameters of the image capturing device.
The image acquisition equipment in the embodiment of the invention can be a common panoramic camera for acquiring panoramic images. Typically a common commercial panoramic camera, or a special device that can capture depth data and RGB image data.
The face recognition device in the embodiment of the invention can be a commonly used mobile device with a face recognition function, such as a mobile phone, a tablet computer, and the like. Any device having a face recognition function may be used as the face recognition device.
The face recognition device in the embodiment of the invention also needs to be provided with virtual reality (AR) technology, and can be applied to an augmented reality software toolkit such as AR-core or AR-kit to perform corresponding operation. The augmented reality software toolkits can be used for acquiring pose parameters of the face recognition equipment and performing subsequent operation.
In the embodiment of the present invention, how to acquire panoramic information by using a conventional face recognition device and an image acquisition device is described by taking face recognition as an example, and the present invention is not limited to face information. In fact, as long as the reference object can be identified by the face identification device and the image acquisition device, and further can perform relative angle calculation, the embodiment of the present invention is not limited to face identification.
As shown in fig. 1, a flow chart of the panoramic information collection pose acquisition principle provided in this embodiment is provided, wherein,
and 11, matching the face information acquired by the face recognition device in the panoramic image acquired by the image acquisition device.
In one embodiment of the present invention, the relative positions of the face recognition device and the image capture device need to be fixed. For example, a tripod can be used, the image capturing device being fixed to the top of the tripod, and the face recognition device being fixed in the middle of the tripod, i.e. below the image capturing device. Therefore, after the tripod moves to the position, the relative positions of the image acquisition equipment and the face recognition equipment cannot be changed.
In one embodiment of the present invention, after the face recognition device captures the face information, the image capture device also captures the panoramic image. Face information should also be included in the panoramic image. By matching the two images, the relative position of specific face information in the panoramic image can be known.
In an embodiment of the present invention, the face recognition device may collect the face information first, and then perform the panoramic image collection of the image collection device after the face information is recognized. The image capturing device may capture a panoramic image while the face recognition device captures face information.
And step 12, obtaining a relative included angle between the face recognition device and the image acquisition device according to the position of the face information in the panoramic image.
In one embodiment of the present invention, the position of the face information in the panoramic image is acquired, and the same face information is determined. Since the face recognition apparatus needs to face the relevant face when recognizing the face information, its angle can be determined. And the image acquisition equipment acquires panoramic information and has no angle. If the orientation of the face recognition device is determined, the position of the face information in the panoramic image can be obtained, that is, a relative angle between the orientation of the face recognition device and the main direction of the image acquisition device on a plane is obtained, and the relative angle is an angle between the face recognition device and the main direction of the image acquisition device to confirm the relative deviation angle between the face recognition device and the main direction of the image acquisition device.
And step 13, acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle.
The pose parameters are the parameters of the position and the posture of the object and are used for representing the position and the posture of the specific object. Pose refers to the problem of estimating camera motion in a number of images given. The pose parameters of the camera can be acquired by a feature point method. Extracting a plurality of features from the images, then carrying out feature matching between the images to obtain a plurality of matched points, and then solving the camera pose according to the points.
In one embodiment of the invention, an augmented reality software toolkit such as an ar-core or an ar-kit can be arranged inside the face recognition device to perform corresponding operations. The augmented reality software toolkits can sense the position and the posture of the face recognition equipment and acquire the pose parameters of the face recognition equipment.
For example, ar-core is a software platform that builds augmented reality applications, introduced by google, and like ar-kit, introduced by apple Inc., can take advantage of advances in cloud software and device hardware to place digital objects in the real world. The main functions of the ar-core include: 1) motion capture: by using the sensor and the camera, the position and the posture of the equipment can be accurately sensed, and the position and the posture of the displayed virtual object can be changed; 2) environmental perception: sensing plane, such as your desk, ground, accurately reproducing this plane in virtual space 3) light source sensing: the environment light sensor of the mobile phone is used for sensing the environment illumination condition, and correspondingly adjusting the brightness, the shadow and the material of the virtual object, so that the virtual object looks more integrated into the environment.
After the pose parameters of the face recognition equipment are obtained, the pose parameters of the image acquisition equipment can be obtained through calculation by combining the relative included angles, and further preparation is made for splicing the panoramic images acquired by the image acquisition equipment.
In one embodiment of the present invention, the fixing of the relative position setting of the face recognition device and the image acquisition device includes:
the relative position, the relative distance and the relative angle of the face recognition equipment and the image acquisition equipment are all fixedly arranged.
The relative position, relative distance, and relative angle of the face recognition device and the image capture device need to be fixed to calculate the determined relative angle.
In one embodiment of the present invention, when the face recognition device acquires the face information, the image capture device performs panoramic image capture.
Since it is required that the face is directed to the face recognition apparatus at the time of face information recognition, face information recognition can be performed more efficiently when the face is directed to the apparatus. At this time, it is determined that the face is facing the face recognition device, and the face recognition device notifies the image capture device to capture an image, so that it can be ensured that the panoramic image captured at this time contains corresponding face information.
In one embodiment of the present invention, the face recognition apparatus acquires any one of face information as the face information.
The face recognition apparatus does not have fixed face information for recognition, but can acquire any face information for recognition. As long as this facial information can be acquired by the image acquisition device at the same time after the recognition, the subsequent operation can be performed.
In an embodiment of the present invention, after the face recognition device obtains the face information, when it is determined that the sharpness and the angle of the face information meet a preset threshold, the face recognition device notifies the image capture device to capture a panoramic image. The face recognition device needs to complete the process of informing the image acquisition device within a short time after acquiring the face information so as to avoid the problem that the face information acquired by the face recognition device cannot be matched with the panoramic image acquired by the image acquisition device.
Or when the face recognition device acquires the face information, the image acquisition device acquires a panoramic image to ensure that the face information acquired by the face recognition device is matched with the panoramic image acquired by the image acquisition device.
The preset threshold value is set according to definition and an angle range required by clear facial information identification.
The premise of face recognition is that the corresponding face information must be clear and complete, and the side face information is difficult to recognize. Thus, a threshold value needs to be set for determining whether the corresponding face information can be accurately recognized.
Generally, a face recognition frame can be set, and when the corresponding face appears in the frame and covers a certain proportion range of the frame, the face can be accurately recognized.
In an embodiment of the present invention, the obtaining a relative angle between the face recognition device and the image capture device according to the position of the face information in the panoramic image includes:
comparing the face information in the panoramic image;
determining the relative position of the face information and an image acquisition device according to the position of the face information in the panoramic image;
and calculating to obtain a relative included angle between the face recognition equipment and the image acquisition equipment according to the relative position between the face information and the image acquisition equipment.
In an embodiment of the present invention, the acquiring the pose parameter of the image capturing device according to the pose parameter of the face recognition device and the relative included angle includes:
acquiring pose parameters of the face recognition equipment;
and obtaining the pose parameters of the image acquisition equipment by combining the relative included angle adjustment according to the pose parameters of the face recognition equipment.
Specifically, in the embodiment of the invention, by taking a face (human face) as an intermediate marker, the relative included angle is calculated by matching the facial information acquired by the face identification device and the facial information in the panoramic image, and the pose of the image acquisition device is obtained, so that the acquisition and the splicing of the spatial information are realized.
The camera on the facial recognition equipment recognizes the facial information and ensures that the facial information accounts for the ratio in the viewing frame, so that the facial information is ensured to be just opposite to the camera, and at the moment, the included angle between the facial recognition equipment and the facial information is 0. If the face information is detected at the moment, recording the face information and sending a shooting instruction to the image acquisition equipment, wherein the position of the person corresponding to the face information is not fixed at the moment. After acquiring a panoramic image shot by image acquisition equipment, identifying the position of the face information in the panoramic image at the moment, namely the relative included angle between the face information and the image acquisition equipment. After the relative included angle is known, the real pose of the image acquisition equipment can be calculated by using the included angle and the pose included angle returned by the virtual reality engine, so that the panoramic images acquired by the image acquisition equipment at different positions are accurately spliced.
According to the scheme of the embodiment of the invention, the face information acquired by the face recognition equipment is matched in the panoramic image acquired by the image acquisition equipment; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions; obtaining a relative included angle between the face recognition equipment and the image acquisition equipment according to the position of the face information in the panoramic image; and acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle. According to the scheme of the embodiment of the invention, the interaction between the face recognition equipment and the image acquisition equipment is utilized, after the face recognition equipment acquires face information, the face information is compared with a panoramic image acquired by the image acquisition equipment to acquire the position in the panoramic image, and then the relative included angle between the face recognition equipment and the image acquisition equipment is determined. And after the face recognition equipment acquires the self pose parameters, acquiring the pose parameters of the image acquisition equipment according to the relative included angle. The pose parameters of the image acquisition equipment are acquired according to the pose parameters of the face recognition equipment in combination with the relative included angles, so that the problem of angle deviation caused by the fact that panoramic images acquired by the image acquisition equipment are spliced due to the fact that no pose parameters exist is solved.
For example, in general, when a panoramic image of a room is acquired, a plurality of panoramic images are acquired at a plurality of positions by a panoramic camera, and then the panoramic images are spliced to obtain an overall model of the room. However, the panoramic camera cannot directly acquire the pose parameters of the panoramic camera, and the pose of the panoramic camera is changed, so that the angles of panoramic images during splicing have deviation, and the panoramic images cannot be effectively spliced. According to the scheme of the embodiment, the pose parameters of the panoramic camera (image acquisition equipment) can be deduced by using the pose parameters acquired by the face recognition equipment, a determined angle is provided for subsequent panoramic image splicing, and panoramic images acquired by the image acquisition equipment at various positions in a room can be automatically spliced into three-dimensional images conveniently.
An embodiment of the present invention provides a panoramic information collection pose acquisition apparatus, as shown in fig. 2, the apparatus including:
a matching unit 21 for matching the face information acquired by the face recognition device in the panoramic image acquired by the image pickup device; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions;
a relative angle obtaining unit 22, configured to obtain a relative included angle between the face recognition device and the image acquisition device according to the position of the face information in the panoramic image;
a pose parameter acquiring unit 23, configured to acquire a pose parameter of the image capturing device according to the pose parameter of the face recognition device and the relative included angle.
In one embodiment of the invention, the apparatus further comprises:
a face recognition unit 24 for acquiring any one of face information as the face information, and transmitting the matching unit 21 to match in the panoramic image acquired by the image pickup device.
In an embodiment of the present invention, the face recognition unit 24 is further configured to notify the image capturing device to perform panoramic image capturing when it is determined that the sharpness and the angle of the face information meet preset threshold values after the face information is acquired; the preset threshold value is set according to definition and an angle range required by clear facial information identification.
The face recognition device needs to complete the process of informing the image acquisition device within a short time after acquiring the face information so as to avoid the problem that the face information acquired by the face recognition device cannot be matched with the panoramic image acquired by the image acquisition device.
Or when the face recognition device acquires the face information, the image acquisition device acquires a panoramic image to ensure that the face information acquired by the face recognition device is matched with the panoramic image acquired by the image acquisition device.
In one embodiment of the invention, the apparatus further comprises:
and a panoramic image collecting unit 25 for collecting a panoramic image.
The relative angle obtaining unit 22 is specifically configured to:
comparing the face information in the panoramic image; determining the relative position of the face information and an image acquisition device according to the position of the face information in the panoramic image; and calculating to obtain a relative included angle between the face recognition equipment and the image acquisition equipment according to the relative position between the face information and the image acquisition equipment.
The pose parameter acquiring unit 23 is specifically configured to:
acquiring pose parameters of the face recognition equipment; and obtaining the pose parameters of the image acquisition equipment by combining the relative included angle adjustment according to the pose parameters of the face recognition equipment.
As shown in fig. 3, an embodiment of the present invention further provides a panoramic information collection pose acquisition system, including a control device 31, a face recognition device 32 and an image collection device 33, which are arranged at fixed relative positions, wherein:
the face recognition device 32 for acquiring face information;
the image acquisition device 33 is used for acquiring a panoramic image;
the control device 31 for matching the face information acquired by the face recognition device 32 in the panoramic image acquired by the image pickup device 33; obtaining a relative included angle between the face recognition device 32 and the image acquisition device 33 according to the position of the face information in the panoramic image; and acquiring the pose parameters of the image acquisition equipment 33 according to the pose parameters of the face recognition equipment 32 and the relative included angles.
The face recognition device 32 is further configured to notify the image capturing device 33 to perform panoramic image capturing when the face information is acquired.
In one embodiment of the present invention, the facial recognition device 32 may be a commonly used mobile device with facial recognition function, such as a mobile phone, a tablet computer, etc. Any device having a face recognition function may be used as the face recognition device 32.
In one embodiment of the present invention, the image capturing device 33 may be a general panoramic camera to capture panoramic images. Which may be a common commercial panoramic camera in general, or a special device that may acquire depth data and RGB image data.
In one embodiment of the present invention, the control device 31 may be provided together with the face recognition device 32 or may be provided separately. The control device 31 is a main control device of the entire system, and needs to control the facial recognition device 32 and the image acquisition device 33 and perform related hiccup operation.
In an embodiment of the present invention, there is also provided a computer-readable storage medium storing a computer program for executing the above-mentioned method.
In one embodiment of the present invention, there is also provided an electronic apparatus including:
a processor;
a memory for storing the processor-executable instructions;
the processor is used for reading the executable instructions from the memory and executing the instructions to realize the method.
Fig. 4 is a schematic structural diagram of an embodiment of an electronic device according to the present invention. As shown in fig. 4, the electronic device includes one or more processors and memory.
The processor may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.
The memory may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by a processor to implement the behavior analysis based matching methods of the various embodiments of the invention described above and/or other desired functions.
In one example, the electronic device may further include: an input device and an output device, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device may also include, for example, a keyboard, a mouse, and the like.
The output device may output various information including the determined distance information, direction information, and the like to the outside. The output devices may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.
Of course, for simplicity, only some of the components of the electronic device that are relevant to the present invention are shown in fig. 4, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device may include any other suitable components, depending on the particular application.
In addition to the above-described methods and apparatus, embodiments of the present invention may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform the steps in the behavioral analysis-based matching method according to various embodiments of the present invention described in the above-mentioned part of the present specification.
The computer program product may write program code for carrying out operations for embodiments of the present invention in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Based on the scheme provided by the embodiment of the invention, the method mainly comprises the following beneficial effects:
matching face information acquired by face recognition equipment in a panoramic image acquired by image acquisition equipment; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions; obtaining a relative included angle between the face recognition equipment and the image acquisition equipment according to the position of the face information in the panoramic image; and acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle. According to the scheme of the embodiment of the invention, the interaction between the face recognition equipment and the image acquisition equipment is utilized, after the face recognition equipment acquires face information, the face information is compared with a panoramic image acquired by the image acquisition equipment to acquire the position in the panoramic image, and then the relative included angle between the face recognition equipment and the image acquisition equipment is determined. And after the face recognition equipment acquires the self pose parameters, acquiring the pose parameters of the image acquisition equipment according to the relative included angle. The pose parameters of the image acquisition equipment are acquired according to the pose parameters of the face recognition equipment in combination with the relative included angles, so that the problem of angle deviation caused by the fact that panoramic images acquired by the image acquisition equipment are spliced due to the fact that no pose parameters exist is solved. According to the position and pose parameters of the image acquisition equipment, panoramic images acquired by the image acquisition equipment at all positions in a room can be conveniently and automatically spliced into three-dimensional images, a vectorized extremely-simple three-dimensional model of the room can be deduced by acquiring the panoramic images of the point positions of the room through simple image acquisition equipment, and the requirement of quick and simple full-house reconstruction can be met.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The method and apparatus of the present invention may be implemented in a number of ways. For example, the methods and apparatus of the present invention may be implemented in software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustrative purposes only, and the steps of the method of the present invention are not limited to the order specifically described above unless specifically indicated otherwise. Furthermore, in some embodiments, the present invention may also be embodied as a program recorded in a recording medium, the program including machine-readable instructions for implementing a method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.
The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Claims (15)
1. A panoramic information acquisition pose acquisition method is characterized by comprising the following steps:
matching face information acquired by face recognition equipment in a panoramic image acquired by image acquisition equipment; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions;
obtaining a relative included angle between the face recognition equipment and the image acquisition equipment according to the position of the face information in the panoramic image;
acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle;
wherein, the obtaining of the relative angle between the face recognition device and the image acquisition device according to the position of the face information in the panoramic image comprises:
comparing the face information in the panoramic image;
determining the relative position of the face information and an image acquisition device according to the position of the face information in the panoramic image;
and calculating to obtain a relative included angle between the face recognition equipment and the image acquisition equipment according to the relative position between the face information and the image acquisition equipment.
2. The method of claim 1, wherein the facial recognition device is in a fixed relative position setting with respect to an image capture device, comprising:
the relative position, the relative distance and the relative angle of the face recognition equipment and the image acquisition equipment are all fixedly arranged.
3. The method of claim 1, wherein the method further comprises:
and when the face recognition equipment acquires the face information, the image acquisition equipment acquires a panoramic image.
4. The method of claim 3, wherein the method further comprises:
the face recognition apparatus acquires any one of face information as the face information.
5. The method of claim 3 or 4, further comprising:
after the face recognition device acquires the face information, when the definition and the angle of the face information are determined to accord with preset threshold values, the face recognition device informs the image acquisition device to acquire a panoramic image;
the preset threshold value is set according to definition and an angle range required by clear facial information identification.
6. The method of claim 1, wherein the obtaining the pose parameters of the image capture device from the pose parameters of the face recognition device and the relative angle comprises:
acquiring pose parameters of the face recognition equipment;
and obtaining the pose parameters of the image acquisition equipment by combining the relative included angle adjustment according to the pose parameters of the face recognition equipment.
7. The utility model provides a panorama information acquisition position appearance acquisition device which characterized in that includes:
a matching unit for matching the face information acquired by the face recognition device in the panoramic image acquired by the image acquisition device; the face recognition equipment and the image acquisition equipment are fixedly arranged at relative positions;
a relative angle obtaining unit, configured to obtain a relative included angle between the face recognition device and an image acquisition device according to a position of the face information in the panoramic image, where the face information is compared in the panoramic image; determining the relative position of the face information and an image acquisition device according to the position of the face information in the panoramic image; calculating to obtain a relative included angle between the face recognition device and the image acquisition device according to the relative position between the face information and the image acquisition device;
and the pose parameter acquisition unit is used for acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle.
8. The apparatus of claim 7, wherein the apparatus further comprises:
and the face recognition unit is used for acquiring any face information as the face information and sending the matching unit to match in the panoramic image acquired by the image acquisition equipment.
9. The apparatus according to claim 8, wherein the face recognition unit is further configured to notify the image capturing device to perform panoramic image capturing when it is determined that the sharpness and the angle of the face information meet preset thresholds after the face information is acquired; the preset threshold value is set according to definition and an angle range required by clear facial information identification.
10. The apparatus of claim 7, wherein the apparatus further comprises:
and the panoramic image acquisition unit is used for acquiring panoramic images.
11. The device according to any one of claims 7 to 10, wherein the pose parameter acquiring unit is specifically configured to:
acquiring pose parameters of the face recognition equipment; and obtaining the pose parameters of the image acquisition equipment by combining the relative included angle adjustment according to the pose parameters of the face recognition equipment.
12. The panoramic information collection pose acquisition system is characterized by comprising a face recognition device and an image collection device which are arranged at fixed relative positions, and a control device, wherein:
the face recognition device is used for acquiring face information;
the image acquisition equipment is used for acquiring a panoramic image;
the control equipment is used for matching the face information acquired by the face recognition equipment in the panoramic image acquired by the image acquisition equipment; obtaining a relative included angle between the face recognition device and the image acquisition device according to the position of the face information in the panoramic image, wherein the face information is compared in the panoramic image; determining the relative position of the face information and an image acquisition device according to the position of the face information in the panoramic image; calculating to obtain a relative included angle between the face recognition device and the image acquisition device according to the relative position between the face information and the image acquisition device; and acquiring the pose parameters of the image acquisition equipment according to the pose parameters of the face recognition equipment and the relative included angle.
13. The system of claim 12, wherein the facial recognition device is further configured to notify the image capture device for panoramic image capture when the facial information is obtained.
14. A computer-readable storage medium, in which a computer program is stored, characterized in that the computer program is adapted to perform the method of any of the preceding claims 1-6.
15. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010966678.0A CN112073640B (en) | 2020-09-15 | 2020-09-15 | Panoramic information acquisition pose acquisition method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010966678.0A CN112073640B (en) | 2020-09-15 | 2020-09-15 | Panoramic information acquisition pose acquisition method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112073640A CN112073640A (en) | 2020-12-11 |
| CN112073640B true CN112073640B (en) | 2022-03-29 |
Family
ID=73695811
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010966678.0A Active CN112073640B (en) | 2020-09-15 | 2020-09-15 | Panoramic information acquisition pose acquisition method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112073640B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115937722A (en) * | 2021-09-30 | 2023-04-07 | 华为技术有限公司 | Equipment positioning method, equipment and system |
| CN115426488B (en) * | 2022-11-04 | 2023-01-10 | 中诚华隆计算机技术有限公司 | Virtual reality image data transmission method, system and chip |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108594999A (en) * | 2018-04-20 | 2018-09-28 | 北京京东金融科技控股有限公司 | Control method and device for panoramic picture display systems |
| CN111629242A (en) * | 2020-05-27 | 2020-09-04 | 腾讯科技（深圳）有限公司 | Image rendering method, device, system, equipment and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109427040B (en) * | 2017-08-22 | 2023-10-10 | 富联国基(上海)电子有限公司 | Image processing apparatus and method |
| TWI649721B (en) * | 2017-12-06 | 2019-02-01 | 凌通科技股份有限公司 | Panoramic photographing method of unmanned aerial vehicle and unmanned aerial vehicle using same |
| CN108168458A (en) * | 2017-12-29 | 2018-06-15 | 北京领航视觉科技有限公司 | A kind of interior three-dimensional data automatic acquisition equipment |
| CN109389367B (en) * | 2018-10-09 | 2021-06-18 | 苏州科达科技股份有限公司 | Personnel attendance checking method, device and storage medium |
| CN111080804B (en) * | 2019-10-23 | 2020-11-06 | 贝壳找房(北京)科技有限公司 | Three-dimensional image generation method and device |
| CN111612842B (en) * | 2020-05-29 | 2023-08-18 | 如你所视(北京)科技有限公司 | Method and device for generating pose estimation model |
-
2020
- 2020-09-15 CN CN202010966678.0A patent/CN112073640B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108594999A (en) * | 2018-04-20 | 2018-09-28 | 北京京东金融科技控股有限公司 | Control method and device for panoramic picture display systems |
| CN111629242A (en) * | 2020-05-27 | 2020-09-04 | 腾讯科技（深圳）有限公司 | Image rendering method, device, system, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112073640A (en) | 2020-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11145038B2 (en) | Image processing method and device for adjusting saturation based on depth of field information | |
| US10002463B2 (en) | Information processing apparatus, information processing method, and storage medium, for enabling accurate detection of a color | |
| JP6560480B2 (en) | Image processing system, image processing method, and program | |
| US10410089B2 (en) | Training assistance using synthetic images | |
| JP5740884B2 (en) | AR navigation for repeated shooting and system, method and program for difference extraction | |
| JP6428266B2 (en) | COLOR CORRECTION DEVICE, COLOR CORRECTION METHOD, AND COLOR CORRECTION PROGRAM | |
| US9129435B2 (en) | Method for creating 3-D models by stitching multiple partial 3-D models | |
| CN111783820A (en) | Image annotation method and device | |
| CN108200334B (en) | Image shooting method and device, storage medium and electronic equipment | |
| WO2016029939A1 (en) | Method and system for determining at least one image feature in at least one image | |
| US20180357819A1 (en) | Method for generating a set of annotated images | |
| KR101923177B1 (en) | Appratus and method for providing augmented reality information based on user | |
| WO2021136386A1 (en) | Data processing method, terminal, and server | |
| US20220078385A1 (en) | Projection method based on augmented reality technology and projection equipment | |
| KR101510312B1 (en) | 3D face-modeling device, system and method using Multiple cameras | |
| CN111970454B (en) | Shot picture display method, device, equipment and storage medium | |
| CN112073640B (en) | Panoramic information acquisition pose acquisition method, device and system | |
| JP2020526735A (en) | Pupil distance measurement method, wearable eye device and storage medium | |
| US10127424B2 (en) | Image processing apparatus, image processing method, and image processing system | |
| JP2016134803A (en) | Image processor and image processing method | |
| CN109934873B (en) | Method, device and equipment for acquiring marked image | |
| CN114640833A (en) | Projection picture adjusting method and device, electronic equipment and storage medium | |
| WO2020032254A1 (en) | Attention target estimating device, and attention target estimating method | |
| CN110310325B (en) | Virtual measurement method, electronic device and computer readable storage medium | |
| US20220303460A1 (en) | Systems and Methods for Generating Consistent Images of Objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |