CN113329179B - Shooting alignment method, device, equipment and storage medium - Google Patents
Shooting alignment method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113329179B CN113329179B CN202110599186.7A CN202110599186A CN113329179B CN 113329179 B CN113329179 B CN 113329179B CN 202110599186 A CN202110599186 A CN 202110599186A CN 113329179 B CN113329179 B CN 113329179B
- Authority
- CN
- China
- Prior art keywords
- picture
- coordinates
- camera
- information
- reference picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Studio Devices (AREA)
Abstract
Description
技术领域technical field
本申请属于拍摄领域,具体涉及一种拍摄对位方法、装置、设备及存储介质。The present application belongs to the field of photographing, and in particular relates to a photographing alignment method, device, equipment and storage medium.
背景技术Background technique
在对电子设备的拍摄效果进行测试时,往往需要进行对位,即将电子设备移动到指定位置进行拍摄。When testing the shooting effect of the electronic device, it is often necessary to perform alignment, that is, to move the electronic device to a designated position for shooting.
通常是将电子设备固定在机械臂的夹具末端,通过控制机械臂的运动来将电子设备移动到指定位置,以实现自动化的拍照测试。电子设备在对拍摄对象进行拍摄时对构图有一定的要求,因此,在进行自动化的拍照测试过程中,需要有比较好的拍摄对位方法来保证满足构图要求进行拍摄。Usually, the electronic device is fixed at the end of the jig of the mechanical arm, and the electronic device is moved to a designated position by controlling the movement of the mechanical arm to realize automated photo testing. Electronic devices have certain requirements for composition when shooting objects. Therefore, in the process of automated photo testing, a better shooting alignment method is required to ensure that the composition requirements are met for shooting.
但是,在多场景和多相机的情况下还没有有效的拍摄对位方案。However, there is no effective shooting alignment scheme in the case of multiple scenes and multiple cameras.
发明内容Contents of the invention
本申请实施例的目的是提供一种拍摄对位方法、装置、设备及存储介质,能够解决目前没有适用于多场景和多相机的拍摄对位方案的问题。The purpose of the embodiments of the present application is to provide a shooting alignment method, device, device and storage medium, which can solve the problem that there is currently no shooting alignment solution suitable for multiple scenes and multiple cameras.
第一方面,本申请实施例提供了一种拍摄对位方法,应用于控制设备,所述方法包括:In the first aspect, the embodiment of the present application provides a shooting alignment method, which is applied to a control device, and the method includes:
获取第一图片以及预存储的参考图片,所述第一图片为第一电子设备被机械臂控制在第一位姿下拍摄的图片,所述参考图片为对目标场景进行拍摄得到的图片;Acquiring a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device controlled by the mechanical arm in the first position, and the reference picture is a picture obtained by shooting a target scene;
确定在拍摄所述参考图片时的第一相机外参信息,以及确定在拍摄所述第一图片时的第二相机外参信息;determining first camera extrinsic information when taking the reference picture, and determining second camera extrinsic information when taking the first picture;
根据所述第一相机外参信息以及所述第二相机外参信息,确定所述机械臂的运动控制参数;determining motion control parameters of the robotic arm according to the first camera extrinsic information and the second camera extrinsic information;
控制所述机械臂按照所述运动控制参数进行运动,其中,所述第一电子设备随着所述机械臂从所述第一位姿运动至第二位姿,以使所述第一电子设备在所述第二位姿拍摄出与所述参考图片的构图匹配的第二图片。controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the first electronic device A second picture matching the composition of the reference picture is photographed at the second pose.
第二方面,本申请实施例提供了一种拍摄对位装置,应用于控制设备,所述装置包括:In the second aspect, the embodiment of the present application provides a shooting alignment device, which is applied to a control device, and the device includes:
获取模块,用于获取第一图片以及预存储的参考图片,所述第一图片为第一电子设备被机械臂控制在第一位姿下拍摄的图片,所述参考图片为对目标场景进行拍摄得到的图片;An acquisition module, configured to acquire a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device under the control of the mechanical arm in the first position, and the reference picture is a picture taken of the target scene get the picture;
第一确定模块,用于确定在拍摄所述参考图片时的第一相机外参信息,以及确定在拍摄所述第一图片时的第二相机外参信息;A first determining module, configured to determine the first camera extrinsic information when taking the reference picture, and determine the second camera extrinsic information when taking the first picture;
第一确定模块,用于根据所述第一相机外参信息以及所述第二相机外参信息,确定所述机械臂的运动控制参数;A first determining module, configured to determine motion control parameters of the robotic arm according to the first camera extrinsic parameter information and the second camera extrinsic parameter information;
控制模块,用于控制所述机械臂按照所述运动控制参数进行运动,其中,所述第一电子设备随着所述机械臂从所述第一位姿运动至第二位姿,以使所述第一电子设备在所述第二位姿拍摄出与所述参考图片的构图匹配的第二图片。A control module, configured to control the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the The first electronic device photographs a second picture matching the composition of the reference picture in the second pose.
第三方面,本申请实施例提供了一种控制设备,该控制设备包括处理器、存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令,所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤。In a third aspect, the embodiment of the present application provides a control device, which includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor, and the program or instruction is controlled by The processor implements the steps of the method described in the first aspect when executed.
第四方面,本申请实施例提供了一种可读存储介质,所述可读存储介质上存储程序或指令,所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤。In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .
第五方面,本申请实施例提供了一种芯片,所述芯片包括处理器和通信接口,所述通信接口和所述处理器耦合,所述处理器用于运行程序或指令,实现如第一方面所述的方法。In the fifth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect the method described.
在本申请实施例中,在对第一电子设备的拍摄效果进行测试的过程中,获取第一电子设备被机械臂控制在第一位姿下拍摄的第一图片以及预存储的参考图片,参考图片为对目标场景进行拍摄得到的图片;然后,获取在拍摄参考图片时的第一相机外参信息以及在拍摄第一图片时的第二相机外参信息,并根据第一相机外参信息和第二相机外参信息确定运动控制参数;再然后,控制机械臂按照运动控制参数进行运动。在机械臂按照运动控制参数完成运动的情况下,第一电子设备随着机械臂的运动从第一位姿运动至第二位姿,第一电子设备在第二位姿下可以拍摄出与参考图片的构图匹配的图片。如此,如果需要对电子设备在多个场景的拍摄效果进行测试,则提供对各个场景拍摄出的参考图片,即可完成对应场景的拍摄效果测试。另外,本申请实施例可以适用于不同相机的拍摄效果测试,具体可适配于广角,x1、x2、x5等多种相机镜头的测试。如此,满足了多场景和多相机的需求。In the embodiment of the present application, in the process of testing the shooting effect of the first electronic device, the first picture taken by the first electronic device controlled by the mechanical arm in the first position and the pre-stored reference picture are acquired, refer to The picture is a picture obtained by shooting the target scene; then, the first camera extrinsic information when taking the reference picture and the second camera extrinsic information when taking the first picture are obtained, and according to the first camera extrinsic information and The extrinsic information of the second camera determines the motion control parameters; then, the mechanical arm is controlled to move according to the motion control parameters. When the mechanical arm completes the movement according to the motion control parameters, the first electronic device moves from the first pose to the second pose with the movement of the robotic arm, and the first electronic device can take pictures in the second pose and the reference The composition of the picture matches the picture. In this way, if it is necessary to test the shooting effect of the electronic device in multiple scenes, the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. In addition, the embodiment of the present application can be applied to the shooting effect test of different cameras, and specifically can be adapted to the test of wide-angle, x1, x2, x5 and other camera lenses. In this way, the requirements of multiple scenes and multiple cameras are met.
附图说明Description of drawings
图1是本申请提供的一种拍摄对位系统的一个实施例的示意图。FIG. 1 is a schematic diagram of an embodiment of a shooting alignment system provided by the present application.
图2是本申请提供的一种拍摄对位方法的一个实施例的流程示意图。FIG. 2 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application.
图3是本申请提供的一种对位标的一个实施例的示意图。Fig. 3 is a schematic diagram of an embodiment of an alignment mark provided by the present application.
图4是本申请提供的一种参考图片的一个实施例的示意图。Fig. 4 is a schematic diagram of an embodiment of a reference picture provided by the present application.
图5是本申请提供的一种棋盘格的一个实施例的示意图。Fig. 5 is a schematic diagram of an embodiment of a checkerboard provided by the present application.
图6示出了本申请提供的世界坐标到图像坐标的映射关系的示意图。FIG. 6 shows a schematic diagram of the mapping relationship between world coordinates and image coordinates provided by the present application.
图7示出了本申请提供的像素坐标、内参矩阵、尺度系统、外参矩阵以及世界坐标之间关系的示意图。FIG. 7 shows a schematic diagram of the relationship among pixel coordinates, internal reference matrix, scale system, external reference matrix and world coordinates provided by the present application.
图8示出了本申请提供的构建尺度空间的示意图。FIG. 8 shows a schematic diagram of constructing a scale space provided by the present application.
图9示出了本申请提供的特征点定位的示意图。FIG. 9 shows a schematic diagram of feature point positioning provided by the present application.
图10示出了本申请提供的特征点主方向分配的示意图。FIG. 10 shows a schematic diagram of the main direction assignment of feature points provided by the present application.
图11是本申请提供的一种拍摄对位方法的一个实施例的流程示意图。FIG. 11 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application.
图12是本申请提供的一种拍摄对位装置的一个实施例的结构示意图。FIG. 12 is a schematic structural diagram of an embodiment of a shooting alignment device provided by the present application.
图13是本申请提供的一种控制设备的一个实施例的结构示意图。Fig. 13 is a schematic structural diagram of an embodiment of a control device provided by the present application.
图14是本申请提供的一种控制设备的另一个实施例的结构示意图。Fig. 14 is a schematic structural diagram of another embodiment of a control device provided by the present application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员获得的所有其他实施例,都属于本申请保护的范围。The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象,而不用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施,且“第一”、“第二”等所区分的对象通常为一类,并不限定对象的个数,例如第一对象可以是一个,也可以是多个。此外,说明书以及权利要求中“和/或”表示所连接对象的至少其中之一,字符“/”,一般表示前后关联对象是一种“或”的关系。The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.
在对本申请提供的拍摄对位方法进行说明之前,先对本申请涉及的名词进行说明。Before describing the photographing alignment method provided in this application, the nouns involved in this application will be described first.
世界坐标系(world coordinate system):用户定义的三维世界的坐标系,为了描述目标物在真实世界里的位置而被引入。World coordinate system (world coordinate system): The coordinate system of the three-dimensional world defined by the user is introduced to describe the position of the target object in the real world.
像素坐标系(pixel coordinate system):为了描述物体成像后的像点在数字图像上(相片)的坐标而引入,是我们真正从相机内读取到的信息所在的坐标系。单位为个(像素数目)。Pixel coordinate system (pixel coordinate system): Introduced to describe the coordinates of the image point of the object on the digital image (photo) after imaging, it is the coordinate system where the information we actually read from the camera is located. The unit is a (number of pixels).
图像坐标系(image coordinate system):为了描述成像过程中物体从相机坐标系到图像坐标系的投影透射关系而引入,方便进一步得到像素坐标系下的坐标。Image coordinate system (image coordinate system): Introduced to describe the projection and transmission relationship of objects from the camera coordinate system to the image coordinate system during the imaging process, so as to facilitate further obtaining the coordinates in the pixel coordinate system.
相机坐标系(camera coordinate system):在相机上建立的坐标系,为了从相机的角度描述物体位置而定义,作为沟通世界坐标系和图像/像素坐标系的中间一环。Camera coordinate system (camera coordinate system): The coordinate system established on the camera is defined to describe the position of the object from the perspective of the camera, as an intermediate link between the world coordinate system and the image/pixel coordinate system.
下面结合附图,通过具体的实施例及其应用场景对本申请实施例进行详细地说明。The following describes the embodiments of the present application in detail through specific embodiments and application scenarios in conjunction with the accompanying drawings.
图1是本申请提供的一种拍摄对位系统的一个实施例的示意图。如图1所示,拍摄对位系统包括控制设备01、机械臂02以及电子设备03。控制设备01可以为计算机设备,比如手机、电脑或者控制台等。电子设备03可以为任意具有拍摄功能的设备,比如手机、摄像机、相机等。FIG. 1 is a schematic diagram of an embodiment of a shooting alignment system provided by the present application. As shown in FIG. 1 , the shooting alignment system includes a control device 01 , a
为了实现电子设备03的对位拍摄,先将电子设备03安装在机械臂02的末端,并建立控制设备01与机械臂02以及电子设备03的通信连接。In order to realize the alignment shooting of the
然后,电子设备03拍摄第一图片,并将拍摄的第一图片发送至控制设备01。其中,电子设备03可以在控制设备01的控制下拍摄第一图片,也可以在工作人员的控制下拍摄第一图片。控制设备01接收到电子设备03发送的第一图片之后,根据第一图片确定机械臂02的运动控制参数,控制机械臂02按照运动控制参数进行运动,以实现电子设备03的对位拍摄。Then, the
其中,可以在控制设备01上安装控制软件,通过该软件控制机械臂02运动。Wherein, control software can be installed on the control device 01, and the movement of the
基于上述的拍摄对位系统,本申请提供一种拍摄对位方法,拍摄对位方法可以应用于上述的控制设备01。图2是本申请提供的一种拍摄对位方法的一个实施例的流程示意图。Based on the above shooting alignment system, the present application provides a shooting alignment method, which can be applied to the above-mentioned control device 01 . FIG. 2 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application.
如图2所示,拍摄对位方法包括:As shown in Figure 2, the shooting alignment method includes:
S102,获取第一图片以及预存储的参考图片,第一图片为第一电子设备被机械臂控制在第一位姿下拍摄的图片,参考图片为对目标场景进行拍摄得到的图片。比如,目标场景为想要测试第一电子设备的拍摄效果时,第一电子设备需要拍摄的场景。S102. Acquire a first picture and a pre-stored reference picture. The first picture is a picture taken by the first electronic device under the control of a mechanical arm in a first position, and the reference picture is a picture obtained by taking a target scene. For example, the target scene is a scene that the first electronic device needs to shoot when it wants to test the shooting effect of the first electronic device.
其中,第一位姿是第一电子设备的当前位姿,相应地,第一图片是第一电子设备在当前位姿下拍摄的图片。参考图片可以是第一电子设备需要测试场景的理想构图的图片。为了对第一电子设备进行对位,并拍摄出参考图片的构图效果,则执行S104。Wherein, the first pose is the current pose of the first electronic device, and correspondingly, the first picture is a picture taken by the first electronic device in the current pose. The reference picture may be a picture of an ideal composition of the test scene required by the first electronic device. In order to align the first electronic device and capture the composition effect of the reference picture, S104 is executed.
作为一个示例,S102具体可以包括:在第一电子设备与控制设备建立通信连接的情况下,接收第一电子设备发送的第一图片;以及获取存储在预定位置的参数图片。As an example, S102 may specifically include: when the first electronic device establishes a communication connection with the control device, receiving a first picture sent by the first electronic device; and acquiring a parameter picture stored in a predetermined location.
拍摄对位方法还包括:Shooting alignment methods also include:
S104,获取在拍摄参考图片时的第一相机外参信息,以及获取在拍摄第一图片时的第二相机外参信息。S104. Acquire the extrinsic information of the first camera when the reference picture is taken, and acquire the extrinsic information of the second camera when the first picture is taken.
其中,第一相机外参信息表征世界坐标到第一坐标的转换关系,第二相机外参信息表征世界坐标到第二坐标的转换关系,其中,世界坐标为参考图片上的多个目标点在世界坐标系中的坐标,第一坐标为多个目标点在参考图片上的坐标,第二坐标为多个目标点在第一图片上的坐标。Among them, the first camera extrinsic information represents the transformation relationship from the world coordinates to the first coordinates, and the second camera extrinsic information represents the transformation relation from the world coordinates to the second coordinates. The coordinates in the world coordinate system, the first coordinates are the coordinates of multiple target points on the reference picture, and the second coordinates are the coordinates of multiple target points on the first picture.
作为一个示例,参考图片上的多个目标点可以为参考图片上的角点,比如,多个目标点为参考图片上的4个角点。第一坐标和第二坐标可以为像素坐标。以第一坐标为例,第一坐标包括在横轴方向上从参考图片上的目标点到坐标原点的像素数量,以及在纵轴方向上从参考图片上的目标点到坐标原点的像素数量。As an example, the multiple target points on the reference picture may be corner points on the reference picture, for example, the multiple target points are four corner points on the reference picture. The first coordinates and the second coordinates may be pixel coordinates. Taking the first coordinate as an example, the first coordinate includes the number of pixels from the target point on the reference picture to the coordinate origin on the horizontal axis, and the number of pixels from the target point on the reference picture to the coordinate origin on the vertical axis.
拍摄对位方法还包括:Shooting alignment methods also include:
S106,根据第一相机外参信息以及第二相机外参信息,确定机械臂的运动控制参数;S106. Determine motion control parameters of the robotic arm according to the first camera extrinsic parameter information and the second camera extrinsic parameter information;
S108,控制机械臂按照运动控制参数进行运动,其中,第一电子设备随着机械臂从第一位姿运动至第二位姿,以使第一电子设备在第二位姿拍摄出与参考图片的构图匹配的第二图片。S108, controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the first electronic device takes a picture in the second pose that is consistent with the reference picture The composition matches the second picture.
本申请实施例中的拍摄对位是依据预先存储的参考图片和实际拍摄的第一图片完成的,参考图片内的物体和场景可以进行替换,对固定场景无固定要求。因此,如果需要对第一电子设备在多个场景(比如近拍和远拍)的拍摄效果进行测试,则提供对各个场景拍摄出的参考图片,即可完成对应场景的拍摄效果测试,让多个场景的快速切换测试成为可能。另外,本申请实施例可以适用于不同相机的拍摄效果测试,比如,在仅使用待测单目摄像头,不借助外部辅助拍摄定位设备的情况下,此申请可适配于广角,x1、x2、x5等多种相机镜头的繁复测试场景。如此,满足了多场景和多相机的需求。而且,本申请实施例针对不固定被拍摄物体,可适用于任意物体的拍照位姿对位,所拍摄图片可以较大程度上满足预想的参考图片的构图。The shooting alignment in the embodiment of the present application is completed based on the pre-stored reference picture and the first picture actually taken. Objects and scenes in the reference picture can be replaced, and there is no fixed requirement for fixed scenes. Therefore, if it is necessary to test the shooting effect of the first electronic device in multiple scenes (such as close-up shooting and long-distance shooting), the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. It is possible to quickly switch between different scenarios for testing. In addition, the embodiment of the present application can be applied to the shooting effect test of different cameras. For example, when only the monocular camera to be tested is used without external auxiliary shooting and positioning equipment, this application can be adapted to wide-angle, x1, x2, Complicated test scenarios for multiple camera lenses such as x5. In this way, the requirements of multiple scenes and multiple cameras are met. Moreover, the embodiment of the present application is aimed at unfixed objects to be photographed, and is applicable to photographing pose alignment of any object, and the photographed pictures can satisfy the expected composition of the reference picture to a large extent.
在本申请的一个或多个实施例中,参考图片中包括至少两个定位标。作为一个示例,定位标可以为如图3所示的定位标,该定位标是一种圆形黑白标志。In one or more embodiments of the present application, the reference picture includes at least two positioning markers. As an example, the positioning mark may be the positioning mark as shown in FIG. 3 , and the positioning mark is a circular black and white mark.
在拍摄参考图片的过程中,可以将定位标放置在拍摄对象的拍摄范围内,如此,拍摄出的参考图片中不仅包括拍摄对象,还包括定位标。比如,参考图片为如图4所示的图片,在图4中,两个定位标放置在光盘的两侧,在对光盘进行拍摄时,不仅拍摄到光盘,还拍摄到在两侧放置的定位标。During the process of taking the reference picture, the positioning mark can be placed within the shooting range of the object to be photographed. In this way, the captured reference picture includes not only the object to be photographed, but also the positioning mark. For example, the reference picture is the picture shown in Figure 4. In Figure 4, two positioning markers are placed on both sides of the disc. mark.
S104之前,拍摄对位方法还可以包括:Prior to S104, the shooting alignment method may also include:
获取两个定位标在参考图片上的第一距离,其中,第一距离可以为两个定位标的中心点的距离;Obtain a first distance between the two locators on the reference picture, where the first distance may be the distance between the center points of the two locators;
根据第一距离以及两个定位标的实际空间距离,确定多个目标点在世界坐标系中的世界坐标;Determine the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual space distance between the two positioning marks;
根据预设的相机外参计算算法,对世界坐标、第一坐标、拍摄参考图片的第二电子设备的内参信息以及第二电子设备的相机畸变系数进行计算,得到第一相机外参信息。其中,第一电子设备可以为照相机或者具有拍照功能的终端(比如手机或者电脑)。According to the preset camera extrinsic parameter calculation algorithm, the world coordinates, the first coordinates, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device are calculated to obtain the first camera extrinsic information. Wherein, the first electronic device may be a camera or a terminal with a camera function (such as a mobile phone or a computer).
作为一个示例,获取两个定位标在参考图片上的第一距离,具体可以包括:检测参考图片上的两个定位标;通过图像识别确定每个定位标在参考图像上的中心点;计算两个定位标的中心点在参考图像上的第一距离,其中,第一距离可以为欧式距离。As an example, obtaining the first distance between the two locators on the reference picture may specifically include: detecting the two locators on the reference picture; determining the center point of each locator on the reference image through image recognition; calculating the two A first distance between the center points of each positioning mark on the reference image, where the first distance may be a Euclidean distance.
参考图片上的两个定位标可以是使用目标检测模型检测得到,目标检测模型可以为YOLO(You Only Look Once)神经网络模型,具体地,目标检测模型可以为卷积神经网络模型yolov4-Tiny。目标检测模型可以部署在端侧,目的是在各种复杂场景对定位标进行目标检测和识别。The two locators on the reference picture can be detected by using a target detection model. The target detection model can be a YOLO (You Only Look Once) neural network model. Specifically, the target detection model can be a convolutional neural network model yolov4-Tiny. The target detection model can be deployed on the device side to detect and recognize targets in various complex scenarios.
除了获取两个定位标在参考图片上的第一距离,还需要获取两个定位标的实际空间距离。两个定位标的实际空间距离可以是由测试人员手动测量得到,也可以是通过传感器测量得到。In addition to obtaining the first distance between the two positioning marks on the reference picture, it is also necessary to obtain the actual spatial distance of the two positioning marks. The actual spatial distance between the two positioning marks can be obtained by manual measurement by the tester, or by sensor measurement.
在得到第一距离以及两个定位标的实际空间距离之后,可以根据第一距离以及两个定位标的实际空间距离,确定多个目标点在世界坐标系中的世界坐标。After the first distance and the actual spatial distance of the two locators are obtained, the world coordinates of the multiple target points in the world coordinate system can be determined according to the first distance and the actual spatial distance of the two locators.
作为一个示例,根据第一距离以及两个定位标的实际空间距离,确定参考图片上的多个目标点在世界坐标系中的世界坐标,可以包括:As an example, determining the world coordinates of multiple target points on the reference picture in the world coordinate system according to the first distance and the actual space distance between the two positioning marks may include:
获取多个目标点在参考图片上的第一相对位置信息;Obtaining first relative position information of multiple target points on the reference picture;
根据第一距离与实际空间距离之间的第一比例以及第一相对位置信息,确定多个目标点在实际空间中的第二相对位置信息,其中,第一相对位置信息与第二相对位置信息之间的比例与第一比例一致;According to the first ratio between the first distance and the actual space distance and the first relative position information, determine the second relative position information of multiple target points in the actual space, wherein the first relative position information and the second relative position information The ratio between is consistent with the first ratio;
根据第二相对位置信息确定参考图片上的多个目标点在世界坐标系中的世界坐标。The world coordinates of the multiple target points on the reference picture in the world coordinate system are determined according to the second relative position information.
下面以多个目标点为参考图片上的四个角点为例说明如何确定世界坐标。The following uses multiple target points as the four corner points on the reference image as an example to illustrate how to determine the world coordinates.
继续参考图4所示的参考图片,获取参考图片上的四个角点(即角点A、角点B、角点C和角点D)之间的第一相对位置信息,第一相对位置信息为每个角点相对于其他角点的相对位置信息。比如,第一相对位置信息包括角点B位于角点A的向右方向上的第100个像素的位置,角点C位于角点B的向上方向上的第70个像素的位置,角点D位于角点C的向左方向上的第100个像素的位置,角点A位于角点D的向下方向的第70个像素的位置。Continue to refer to the reference picture shown in FIG. 4 to obtain the first relative position information between the four corner points (ie corner point A, corner point B, corner point C and corner point D) on the reference picture, the first relative position The information is the relative position information of each corner point relative to other corner points. For example, the first relative position information includes the position of corner B at the 100th pixel in the right direction of corner A, the position of corner C at the 70th pixel in the upward direction of corner B, and the position of corner D It is located at the 100th pixel in the leftward direction of corner point C, and the corner point A is located at the 70th pixel in the downward direction of corner point D.
假设两个定位标在参考图片上的第一距离为50个像素,两个定位标的实际空间距离为50厘米,而在参考图片上角点B位于角点A的向右方向上的第100个像素的位置,因此,在实际空间中,角点B位于角点A的向右方向上的100厘米的位置。然后,以参考图片的中心点为坐标原点,建立世界坐标系,根据角点B与角点A在实际空间中的相对位置,可以确定角点B与角点A在世界坐标系中的世界坐标。同理,可以确定参考图片中的角点C与角点D在实际空间中的相对位置,以及角点C与角点D在世界坐标系中的世界坐标。Assume that the first distance between the two positioning marks on the reference picture is 50 pixels, the actual space distance between the two positioning marks is 50 cm, and corner point B is located at the 100th point to the right of corner point A on the reference picture The position of the pixel, therefore, in real space, corner point B is located 100 centimeters to the right of corner point A. Then, take the center point of the reference picture as the coordinate origin to establish a world coordinate system, and according to the relative positions of corner point B and corner point A in the actual space, the world coordinates of corner point B and corner point A in the world coordinate system can be determined . Similarly, the relative positions of the corner point C and the corner point D in the reference picture in the real space, and the world coordinates of the corner point C and the corner point D in the world coordinate system can be determined.
以上是以多个目标点为参考图片上的四个角点为例说明的确定世界坐标的一个示例。其中,在拍摄参考图片时,可以让定位标放置于希望拍摄的场景的任意位置,可以精确计算出拍摄的参考图片上的目标点的世界坐标。The above is an example of determining the world coordinates by taking multiple target points as the four corner points on the reference picture as an example. Wherein, when the reference picture is taken, the positioning mark can be placed at any position of the scene desired to be taken, and the world coordinates of the target point on the taken reference picture can be accurately calculated.
多个目标点的世界坐标用于确定第一相机外参信息,而第一相机外参信息的确定不仅需要用到多个目标点的世界坐标,还需要用到拍摄参考图片的第二电子设备的内参信息。该内参信息可以通过对拍摄参考图片的相机进行标定得到。The world coordinates of multiple target points are used to determine the first camera extrinsic information, and the determination of the first camera extrinsic information not only needs to use the world coordinates of multiple target points, but also requires the use of a second electronic device that takes reference pictures internal reference information. The internal reference information can be obtained by calibrating the camera that takes the reference picture.
作为一个示例,可以使用张正友相机标定法对拍摄参考图片的相机进行标定。其中,可以调用opencv相机标定接口来完成相机内参的标定。As an example, Zhang Zhengyou's camera calibration method can be used to calibrate the camera that takes the reference picture. Among them, the opencv camera calibration interface can be called to complete the calibration of the camera internal parameters.
具体地,先使用该相机对棋盘格进行拍照,得到包括棋盘格的照片。照片的数量取决于内参信息是否收敛到规定误差范围之内,一般至少四张照片,正拍一张,倾斜三张。棋盘格的样式如图5所示。而且,可以允许不拍满整个棋盘格,并可以对所拍摄的照片进行亚像素角点检测,增加了鲁棒性。Specifically, the camera is first used to take pictures of the checkerboard to obtain a photo including the checkerboard. The number of photos depends on whether the internal reference information converges within the specified error range. Generally, at least four photos are taken, one is taken straight and three are tilted. The pattern of the checkerboard is shown in Figure 5. Moreover, it is not allowed to take pictures of the entire checkerboard, and sub-pixel corner detection can be performed on the pictures taken, which increases the robustness.
在拍摄包括棋盘格的照片之后,识别照片中被标记的白格内二维码,白格内的二维码中存储有对应白格在棋盘上的位置信息。具体地,白格内的二维码中存储有一个序号,该序号对应该白格在棋盘上几行几列的位置信息。After the photo including the checkerboard is taken, the QR code in the marked white grid in the photo is recognized, and the position information of the corresponding white grid on the chessboard is stored in the QR code in the white grid. Specifically, a serial number is stored in the two-dimensional code in the white grid, and the serial number corresponds to the position information of the rows and columns of the white grid on the chessboard.
在确定白格在棋盘上的位置信息之后,基于白格在棋盘上的位置信息,可以确定棋盘的位置,然后对棋盘上的黑白角点进行检测,并根据检测结果,确定各角点在照片上的像素坐标以及各角点的世界坐标。After determining the position information of the white grid on the chessboard, based on the position information of the white grid on the chessboard, the position of the chessboard can be determined, and then the black and white corner points on the chessboard are detected, and according to the detection results, it is determined that each corner point is in the photo The pixel coordinates on and the world coordinates of each corner point.
基于各角点在照片上的像素坐标以及各角点的世界坐标,可以确定第二电子设备的内参信息。Based on the pixel coordinates of each corner point on the photo and the world coordinates of each corner point, internal reference information of the second electronic device can be determined.
下面结合图6说明如何确定第二电子设备的内参信息。How to determine the internal reference information of the second electronic device will be described below with reference to FIG. 6 .
图6示出了本申请提供的一种世界坐标到图像坐标的映射关系的示意图。如图6所示,在3D空间中,点在世界坐标系中的世界坐标M=[X,Y,Z]T,其齐次坐标在2D空间中,点在像素坐标系中的坐标m=[u,v]T,其齐次坐标 FIG. 6 shows a schematic diagram of a mapping relationship between world coordinates and image coordinates provided by the present application. As shown in Figure 6, in 3D space, the world coordinate M=[X,Y,Z] T of a point in the world coordinate system, its homogeneous coordinate In 2D space, the coordinate m=[u, v] T of a point in the pixel coordinate system, its homogeneous coordinate
假设棋盘格位于Z=0,定义旋转矩阵R的第i列为ri,则有:Assuming that the checkerboard is located at Z=0, and the i-th column of the rotation matrix R is defined as r i , then:
因此,世界坐标系到图像坐标系的映射为:Therefore, the mapping from the world coordinate system to the image coordinate system is:
基于此,像素坐标、内参矩阵、尺度系统s、外参矩阵以及世界坐标之间的关系如图7所示。Based on this, the relationship among pixel coordinates, internal parameter matrix, scale system s, external parameter matrix and world coordinates is shown in Figure 7.
在确定第二电子设备的内参信息的过程中,需要将一张图中的点映射到另一张图中对应的点,单应性变换是对齐次坐标下点的现行变换,单应性矩阵H约束性强,是点到点的一一对应。H为一个3*3的矩阵,并且有一个元素作为齐次坐标,因此,H有8个自由度。现在有8个自由度需要求解,所以需要四个对应点,也就是说通过四个点就可以求出图像坐标系到世界坐标系的单应性矩阵H,单应性矩阵H中包括内参矩阵和外参矩阵。单应性矩阵H中包括的内参矩阵即为第二电子设备的内参信息。In the process of determining the internal reference information of the second electronic device, it is necessary to map the points in one picture to the corresponding points in another picture. The homography transformation is the current transformation of the points under the aligned sub-coordinates. The homography matrix H is highly restrictive. , is a point-to-point one-to-one correspondence. H is a 3*3 matrix with one element as a homogeneous coordinate, therefore, H has 8 degrees of freedom. Now there are 8 degrees of freedom to be solved, so four corresponding points are needed, that is to say, the homography matrix H from the image coordinate system to the world coordinate system can be obtained through four points, and the homography matrix H includes the internal reference matrix and extrinsic matrix. The internal reference matrix included in the homography matrix H is the internal reference information of the second electronic device.
其中,内参矩阵具体包括一个像素的物理尺寸和焦距,图像物理坐标的扭曲因子,图像原点相对于光心成像点的的纵横偏移量u和v。外参矩阵包括世界坐标系转换到相机坐标系的旋转R和平移T矩阵。Among them, the internal reference matrix specifically includes the physical size and focal length of a pixel, the distortion factor of the physical coordinates of the image, and the vertical and horizontal offsets u and v of the image origin relative to the optical center imaging point. The external parameter matrix includes the rotation R and translation T matrices from the world coordinate system to the camera coordinate system.
此外,由于真实的镜头并非理想的透视成像,而是带有不同程度的畸变。理论上镜头的畸变包括径向畸变和切向畸变。因此,在对相机进行标定的过程中,还可以求出第二电子设备的相机畸变系数,相机畸变系数包括径向相机畸变系数k1,k2,k3,~,和相机的切向相机畸变系数p1,p2,~。由于相机的切向畸变影响较小,通常考虑径向畸变,而且在径向畸变的求解过程中,主要考虑起主导的二元泰勒级数展开的前两个系数k1和k2。In addition, because the real lens is not an ideal perspective imaging, but has different degrees of distortion. Theoretically, lens distortion includes radial distortion and tangential distortion. Therefore, in the process of calibrating the camera, the camera distortion coefficient of the second electronic device can also be obtained, and the camera distortion coefficient includes radial camera distortion coefficients k1, k2, k3, ~, and tangential camera distortion coefficient p1 of the camera , p2, ~. Since the tangential distortion of the camera has less influence, the radial distortion is usually considered, and in the process of solving the radial distortion, the first two coefficients k1 and k2 of the dominant binary Taylor series expansion are mainly considered.
上述已经说明如何确定多个目标点的世界坐标、拍摄参考图片的第二电子设备的内参信息以及第二电子设备的相机畸变系数,而多个目标点在参考图片上的第一坐标可以基于参考图片得到。然后,可以根据预设的相机外参计算算法,对世界坐标、第一坐标以及拍摄参考图片的第二电子设备的内参信息进行计算,得到第一相机外参信息。其中,相机外参计算算法可以为N点透视位姿求解(solve Perspective-N-Point,solvePNP)。The above has explained how to determine the world coordinates of multiple target points, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device, and the first coordinates of multiple target points on the reference picture can be based on the reference picture to get. Then, the world coordinates, the first coordinates, and the internal parameter information of the second electronic device that took the reference picture can be calculated according to the preset camera external parameter calculation algorithm to obtain the first camera external parameter information. Among them, the camera extrinsic parameter calculation algorithm can solve for N-point perspective pose (solve Perspective-N-Point, solvePNP).
在利用solvePNP求解第一相机外参信息时,solvePNP求解依据的相机模型可以包括如图7所示的关系,基于图7所示的关系,对多个目标点在参考图片上的像素坐标(即第一坐标)、拍摄参考图片的相机的内参矩阵、尺度系数以及多个目标点的世界坐标进行计算,得到的外参矩阵即为第一相机外参信息。When using solvePNP to solve the first camera extrinsic information, the camera model based on solvePNP solution can include the relationship shown in Figure 7, based on the relationship shown in Figure 7, the pixel coordinates of multiple target points on the reference picture (i.e. The first coordinate), the internal parameter matrix of the camera that took the reference picture, the scale coefficient, and the world coordinates of multiple target points are calculated, and the obtained external parameter matrix is the external parameter information of the first camera.
在本申请的一个或多个实施例中,S104之前,拍摄对位方法还可以包括:In one or more embodiments of the present application, before S104, the shooting alignment method may further include:
对参考图片和第一图片进行特征检测处理,得到参考图片到第一图片的转换矩阵;performing feature detection processing on the reference picture and the first picture to obtain a conversion matrix from the reference picture to the first picture;
根据转换矩阵以及第一坐标,进行透视变换,得到第二坐标;Perform perspective transformation according to the transformation matrix and the first coordinate to obtain the second coordinate;
根据预设的相机外参计算算法,对世界坐标、第二坐标、第一电子设备的内参信息以及第一电子设备的相机畸变系数进行计算,得到第二相机外参信息。The world coordinates, the second coordinates, the internal parameter information of the first electronic device, and the camera distortion coefficient of the first electronic device are calculated according to the preset camera external parameter calculation algorithm to obtain the second camera external parameter information.
作为一个示例,参考图片到第一图片的转换矩阵可以是使用加速稳健特征(Speeded Up Robust Features,SURF)算法对参考图片和第一图片进行特征匹配得到。As an example, the transformation matrix from the reference picture to the first picture may be obtained by using a Speeded Up Robust Features (SURF) algorithm to perform feature matching on the reference picture and the first picture.
下面对如何得到转换矩阵H进行示例性说明。How to obtain the conversion matrix H is exemplified below.
1、对参考图片和第一图片中的每个图片构建黑塞矩阵(Hessian),生成图片上的关键点,用于特征的提取。1. Construct a Hessian matrix (Hessian) for each picture in the reference picture and the first picture, and generate key points on the picture for feature extraction.
图片f(x,y)的Hessian矩阵如下:The Hessian matrix of the picture f(x, y) is as follows:
在构造黑塞矩阵前需要对图片进行高斯滤波,经过滤波后的Hessian矩阵表述为:Before constructing the Hessian matrix, it is necessary to perform Gaussian filtering on the picture, and the filtered Hessian matrix is expressed as:
其中,当黑塞矩阵的判别式取局部极大值时,判定当前点是比周围领域内其他点更亮或更暗的点,由此来确定关键点的位置。Among them, when the discriminant of the Hessian matrix takes a local maximum value, it is judged that the current point is brighter or darker than other points in the surrounding area, thereby determining the position of the key point.
2、构建尺度空间2. Construct scale space
在SURF中,不同组间图像的尺寸都是一样的,不同的是不同组间使用的盒式滤波器的模板尺寸逐渐增大,同一组间不同层间使用相同尺寸的滤波器,单是滤波器的模糊系数逐渐增大,如图8所示。In SURF, the size of the image between different groups is the same, the difference is that the template size of the box filter used between different groups gradually increases, and the same size filter is used between different layers in the same group, just filtering The blur coefficient of the filter increases gradually, as shown in Figure 8.
3、特征点定位3. Feature point positioning
参考图9,将经过黑塞矩阵处理的每个像素点与二维图像空间和尺度空间邻域内的26个点进行比较,初步定位出关键点,再经过滤除能量比较弱的关键点以及错误定位的关键点,筛选出最终的稳定的特征点。Referring to Figure 9, each pixel processed by the Hessian matrix is compared with 26 points in the two-dimensional image space and scale space neighborhood, and the key points are initially located, and then the key points with weaker energy and errors are filtered out. Locate the key points and filter out the final stable feature points.
4、特征点主方向分配4. Main direction assignment of feature points
在SURF中,采用的是统计特征点圆形邻域内的哈尔特征(Haar-like features,简称harr)小波特征。即在特征点的圆形邻域内,统计60度扇形内所有点的水平、垂直harr小波特征总和,然后扇形以0.2弧度大小的间隔进行旋转并再次统计该区域内harr小波特征值之后,最后将值最大的那个扇形的方向作为该特征点的主方向。该过程示意图如图10所示。In SURF, the Haar-like features (Haar-like features, referred to as harr) wavelet features in the circular neighborhood of statistical feature points are used. That is, in the circular neighborhood of feature points, the sum of the horizontal and vertical harr wavelet features of all points in the 60-degree fan is counted, and then the fan is rotated at an interval of 0.2 radians and the harr wavelet eigenvalues in the area are counted again, and finally the The direction of the sector with the largest value is taken as the main direction of the feature point. The schematic diagram of the process is shown in Figure 10.
5、生成特征点描述子5. Generate feature point descriptors
在特征点周围取一个4*4的矩形区域块,但是所取得矩形区域方向是沿着特征点的主方向。每个子区域统计25个像素的水平方向和垂直方向的haar小波特征。Take a 4*4 rectangular area block around the feature point, but the direction of the obtained rectangular area is along the main direction of the feature point. Each sub-region counts the haar wavelet features of the horizontal and vertical directions of 25 pixels.
6、特征匹配6. Feature matching
通过计算两个特征点间的欧式距离来确定匹配度,欧氏距离越短,代表两个特征点的匹配度越好。在SURF中,还加入了黑塞矩阵迹的判断,如果两个特征点的矩阵迹正负号相同,代表这两个特征具有相同方向上的对比度变化,如果不同,说明这两个特征点的对比度变化方向是相反的,即使欧氏距离为0,页直接予以排除。因此,根据参考图片中的特征点与第一图片中的特征点之间的欧式距离,生成参考图片到第一图片的转换矩阵。The matching degree is determined by calculating the Euclidean distance between two feature points. The shorter the Euclidean distance, the better the matching degree of the two feature points. In SURF, the judgment of the Hessian matrix trace is also added. If the matrix traces of two feature points have the same sign, it means that the two features have contrast changes in the same direction. If they are different, it means that the two feature points have the same contrast. The direction of contrast change is opposite, even if the Euclidean distance is 0, the page is directly excluded. Therefore, according to the Euclidean distance between the feature points in the reference picture and the feature points in the first picture, a conversion matrix from the reference picture to the first picture is generated.
在得到参考图片到第一图片的转换矩阵之后,根据转换矩阵以及第一坐标,进行透视变换,得到多个目标点在第一图片上的第二坐标。如此,通过透视变换,确定了参考图片上的目标点在第一图片上的位置。After the transformation matrix from the reference picture to the first picture is obtained, perspective transformation is performed according to the transformation matrix and the first coordinates to obtain the second coordinates of the multiple target points on the first picture. In this way, through perspective transformation, the position of the target point on the reference picture on the first picture is determined.
在得到多个目标点的世界坐标、多个目标点在第一图片上的第二坐标、第一电子设备的内参信息以及相机畸变系数之后,可以对这些数据进行计算得到第二相机外参信息。其中,计算第二相机外参信息的方式与计算第一相机外参信息的方式类似,而已经说明如何计算得到第一相机外参信息,在此不再对如何计算得到第二相机外参信息进行类似赘述。After obtaining the world coordinates of multiple target points, the second coordinates of multiple target points on the first picture, the internal parameter information of the first electronic device, and the camera distortion coefficient, these data can be calculated to obtain the external parameter information of the second camera . Among them, the way of calculating the extrinsic information of the second camera is similar to the way of calculating the extrinsic information of the first camera, but how to calculate the extrinsic information of the first camera has been explained, and how to calculate the extrinsic information of the second camera will not be discussed here Do similar repetitions.
其中,第一电子设备的内参信息以及相机畸变系数可以通过对第一电子设备进行标定得到。具体的标定方式与上述对拍摄参考图片的相机的标定方式类似,在此不再进行类似赘述。Wherein, the internal reference information and the camera distortion coefficient of the first electronic device may be obtained by calibrating the first electronic device. The specific calibration method is similar to the above-mentioned calibration method for the camera that captures the reference picture, and will not be repeated here.
在本申请的一个或多个实施例中,第一相机外参信息以及第二相机外参信息均通过旋转向量的形式进行表示。为了便于得到机械臂的运动控制参数,需要对第一相机外参信息以及第二相机外参信息进行旋转矩阵变换。In one or more embodiments of the present application, both the first camera extrinsic parameter information and the second camera extrinsic parameter information are expressed in the form of rotation vectors. In order to obtain the motion control parameters of the manipulator, it is necessary to perform a rotation matrix transformation on the first camera extrinsic parameter information and the second camera extrinsic parameter information.
基于此,S106具体可以包括:Based on this, S106 may specifically include:
将第一相机外参信息以及第二相机外参信息分别转换为旋转矩阵,得到第一相机外参信息对应的第一旋转矩阵,以及第二相机外参信息对应的第二旋转矩阵;converting the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain a first rotation matrix corresponding to the first camera extrinsic information and a second rotation matrix corresponding to the second camera extrinsic information;
根据第一旋转矩阵和第二旋转矩阵,确定机械臂的运动控制参数。According to the first rotation matrix and the second rotation matrix, the motion control parameters of the mechanical arm are determined.
其中,可以根据罗德里格斯公式将旋转向量转换为旋转矩阵,罗德里格斯公式如下:Among them, the rotation vector can be converted into a rotation matrix according to the Rodriguez formula, and the Rodriguez formula is as follows:
R=cosθI+(1-cosθ)nnT+sinθn^ (5)R=cosθI+(1-cosθ)nn T +sinθn^ (5)
其中,I是单位矩阵,n是旋转向量的单位向量,θ是旋转向量的模长。Among them, I is the identity matrix, n is the unit vector of the rotation vector, and θ is the modulus length of the rotation vector.
作为一个示例,运动控制参数包括机械臂的偏移信息以及姿态调整角度;根据第一旋转矩阵和第二旋转矩阵,确定机械臂的运动控制参数,具体可以包括:As an example, the motion control parameters include the offset information of the robotic arm and the attitude adjustment angle; according to the first rotation matrix and the second rotation matrix, determining the motion control parameters of the robotic arm may specifically include:
将第一旋转矩阵与第二旋转矩阵的转置矩阵相乘,得到姿态调整角度,以及将第一旋转矩阵与第二旋转矩阵进行相减,得到偏移信息。The first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and the first rotation matrix is subtracted from the second rotation matrix to obtain offset information.
其中,第一旋转矩阵与第二旋转矩阵的转置矩阵相乘,得到是拍摄第一图片到参考图片的相机旋转矩阵,将该相机旋转矩阵确定为姿态调整欧拉角(即姿态调整角度)。将第一旋转矩阵与第二旋转矩阵进行相减,得到机械臂的位置偏移值。Wherein, the first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the camera rotation matrix from the first picture to the reference picture, and the camera rotation matrix is determined as the attitude adjustment Euler angle (ie attitude adjustment angle) . The first rotation matrix is subtracted from the second rotation matrix to obtain the position offset value of the mechanical arm.
在得到机械臂的上述运动控制参数之后,如果机械臂的夹具末端夹持着第一电子设备,那么在控制机械臂的夹具按照运动控制参数进行运动的情况下,电子设备以及第一电子设备也随着夹具的运动而运动。在机械臂的夹具完成运动的情况下,第一电子设备运动至第二位姿。在第二位姿下,第一电子设备可以拍摄出与参考图片的构图匹配的第二图片。After obtaining the above-mentioned motion control parameters of the mechanical arm, if the end of the clamp of the mechanical arm clamps the first electronic device, then when the clamp of the mechanical arm is controlled to move according to the motion control parameters, the electronic device and the first electronic device also Moves with the movement of the gripper. When the gripper of the mechanical arm completes the motion, the first electronic device moves to the second posture. In the second pose, the first electronic device may take a second picture that matches the composition of the reference picture.
需要说明的是,夹具标定是在机械臂控制那部分,本申请实施例中默认夹具末端为标定好的用户坐标系,可直接通过给机械臂用户坐标系偏移值进行控制。It should be noted that the fixture calibration is in the control part of the robotic arm. In the embodiment of this application, the end of the fixture defaults to the calibrated user coordinate system, which can be directly controlled by giving the robotic arm a user coordinate system offset value.
在本申请实施例中,通过将旋转向量形式的第一相机外参信息以及第二相机外参信息分别转换成对应的旋转矩阵,便于得到机械臂的运动控制参数,实现拍摄对位的精准控制。In the embodiment of the present application, by converting the first camera extrinsic information and the second camera extrinsic information in the form of rotation vectors into corresponding rotation matrices, it is easy to obtain the motion control parameters of the robotic arm and realize precise control of shooting alignment .
下面结合图11对本申请实施例进一步地说明。The embodiment of the present application will be further described below with reference to FIG. 11 .
图11是本申请提供的一种拍摄对位方法的一个实施例的流程示意图。如图11所示,拍摄对位方法包括:FIG. 11 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application. As shown in Figure 11, the shooting alignment method includes:
S202,使用棋盘格对拍摄参考图片的第二电子设备的内参信息以及第二电子设备的相机畸变系数进行标定;S202, use the checkerboard to calibrate the internal reference information of the second electronic device that captures the reference picture and the camera distortion coefficient of the second electronic device;
S204,获取参考图片上的两个定位标在参考图片上的第一距离;S204. Obtain a first distance between two locators on the reference picture on the reference picture;
S206,根据第一距离以及两个定位标的实际空间距离,确定参考图片上的四个角点的世界坐标;S206. Determine the world coordinates of the four corner points on the reference picture according to the first distance and the actual space distance between the two locators;
S208,根据拍摄参考图片的第二电子设备的内参信息、第二电子设备的相机畸变系数、四个角点的世界坐标、四个角点在参考图片上的第一坐标,确定拍摄参考图片的相机的第一相机外参信息;S208, according to the internal reference information of the second electronic device that took the reference picture, the camera distortion coefficient of the second electronic device, the world coordinates of the four corner points, and the first coordinates of the four corner points on the reference picture, determine the location of the reference picture The first camera extrinsic information of the camera;
S210,在机械臂上的第一电子设备拍摄出第一图片的情况下,对参考图片和第一图片进行特征检测处理,得到参考图片到第一图片的转换矩阵;S210. When the first electronic device on the mechanical arm captures the first picture, perform feature detection processing on the reference picture and the first picture to obtain a conversion matrix from the reference picture to the first picture;
S212,根据转换矩阵以及四个角点在参考图片上的第一坐标,进行透视变换,得到四个角点在第一图片上的第二坐标;S212. Perform perspective transformation according to the transformation matrix and the first coordinates of the four corner points on the reference picture to obtain second coordinates of the four corner points on the first picture;
S214,根据第一电子设备的内参信息、第一电子设备的相机畸变系数、四个角点的世界坐标以及四个角点在第一图片上的第二坐标,确定第一电子设备的第二相机外参信息;S214. Determine the second position of the first electronic device according to the internal reference information of the first electronic device, the camera distortion coefficient of the first electronic device, the world coordinates of the four corner points, and the second coordinates of the four corner points on the first picture. Camera extrinsic information;
S216,将由旋转向量表示的第一相机外参信息转换为基于世界坐标系的第一旋转矩阵,将由旋转向量表示的第二相机外参信息转换为基于世界坐标系的第二旋转矩阵;S216, converting the first camera extrinsic information represented by the rotation vector into a first rotation matrix based on the world coordinate system, and converting the second camera extrinsic information represented by the rotation vector into a second rotation matrix based on the world coordinate system;
S218,将第一旋转矩阵与第二旋转矩阵的转置矩阵相乘,得到拍摄第一图片到参考图片的目标旋转矩阵;S218. Multiply the first rotation matrix by the transpose matrix of the second rotation matrix to obtain a target rotation matrix from the first picture to the reference picture;
S220,将目标旋转矩阵确定为机械臂的夹具末端的位置偏移值,以及将第一旋转矩阵与第二旋转矩阵相减,得到机械臂末端的姿态调整欧拉角。S220. Determine the target rotation matrix as the position offset value of the clamp end of the mechanical arm, and subtract the first rotation matrix from the second rotation matrix to obtain the attitude adjustment Euler angles of the mechanical arm end.
与本申请提供的一种拍摄对位方法对应地,本申请还提供一种拍摄对位装置,图12是本申请提供的一种拍摄对位装置的一个实施例的结构示意图。如图12所示,拍摄对位装置300包括:Corresponding to the photographing alignment method provided in the present application, the present application also provides a photographing alignment device. FIG. 12 is a schematic structural diagram of an embodiment of the photographing alignment device provided in the present application. As shown in Figure 12, the photographing
第一获取模块302,用于获取第一图片以及预存储的参考图片,第一图片为第一电子设备被机械臂控制在第一位姿下拍摄的图片,参考图片为对目标场景进行拍摄得到的图片;The first obtaining
第二获取模块304,用于获取在拍摄参考图片时的第一相机外参信息,以及获取在拍摄第一图片时的第二相机外参信息;The second acquiring
第一确定模块306,用于根据第一相机外参信息以及第二相机外参信息,确定机械臂的运动控制参数;The first determining
控制模块308,用于控制机械臂按照运动控制参数进行运动,其中,第一电子设备随着机械臂从第一位姿运动至第二位姿,以使第一电子设备在第二位姿拍摄出与参考图片的构图匹配的第二图片。The
本申请实施例中的拍摄对位是依据预先存储的参考图片和实际拍摄的第一图片完成的,参考图片内的物体和场景可以进行替换,对固定场景无固定要求。因此,如果需要对第一电子设备在多个场景(比如近拍和远拍)的拍摄效果进行测试,则提供对各个场景拍摄出的参考图片,即可完成对应场景的拍摄效果测试,让多个场景的快速切换测试成为可能。另外,本申请实施例可以适用于不同相机的拍摄效果测试。如此,满足了多场景和多相机的需求。而且,本申请实施例针对不固定被拍摄物体,可适用于任意物体的拍照位姿对位,所拍摄图片可以较大程度上满足预想的参考图片的构图。The shooting alignment in the embodiment of the present application is completed based on the pre-stored reference picture and the first picture actually taken. Objects and scenes in the reference picture can be replaced, and there is no fixed requirement for fixed scenes. Therefore, if it is necessary to test the shooting effect of the first electronic device in multiple scenes (such as close-up shooting and long-distance shooting), the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. It is possible to quickly switch between different scenarios for testing. In addition, the embodiment of the present application may be applicable to shooting effect tests of different cameras. In this way, the requirements of multiple scenes and multiple cameras are met. Moreover, the embodiment of the present application is aimed at unfixed objects to be photographed, and is applicable to photographing pose alignment of any object, and the photographed pictures can satisfy the expected composition of the reference picture to a large extent.
在本申请的一个或多个实施例中,第一相机外参信息表征世界坐标到第一坐标的转换关系,第二相机外参信息表征世界坐标到第二坐标的转换关系,其中,世界坐标为参考图片上的多个目标点在世界坐标系中的坐标,第一坐标为多个目标点在参考图片上的坐标,第二坐标为多个目标点在第一图片上的坐标。In one or more embodiments of the present application, the first camera extrinsic information represents the transformation relationship from the world coordinate to the first coordinate, and the second camera extrinsic information represents the transformation relation from the world coordinate to the second coordinate, wherein the world coordinate are the coordinates of multiple target points on the reference picture in the world coordinate system, the first coordinates are the coordinates of the multiple target points on the reference picture, and the second coordinates are the coordinates of the multiple target points on the first picture.
在本申请的一个或多个实施例中,参考图片中包括至少两个定位标;拍摄对位装置300还可以包括:In one or more embodiments of the present application, at least two positioning marks are included in the reference picture; the
第三获取模块,用于获取两个定位标在参考图片上的第一距离;The third obtaining module is used to obtain the first distance between the two positioning markers on the reference picture;
第二确定模块,用于根据第一距离以及两个定位标的实际空间距离,确定多个目标点在世界坐标系中的世界坐标;The second determination module is used to determine the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual spatial distance of the two positioning marks;
第一计算模块,用于根据预设的相机外参计算算法,对世界坐标、第一坐标、拍摄参考图片的第二电子设备的内参信息以及第二电子设备的相机畸变系数进行计算,得到第一相机外参信息。The first calculation module is used to calculate the world coordinates, the first coordinates, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device according to the preset camera external parameter calculation algorithm, to obtain the second electronic device. A camera extrinsic information.
在本申请的一个或多个实施例中,第二确定模块可以包括:In one or more embodiments of the present application, the second determination module may include:
第一获取单元,用于获取多个目标点在参考图片上的第一相对位置信息;A first acquiring unit, configured to acquire first relative position information of a plurality of target points on a reference picture;
第一确定单元,用于根据第一距离与实际空间距离之间的第一比例以及第一相对位置信息,确定多个目标点在实际空间中的第二相对位置信息,其中,第一相对位置信息与第二相对位置信息之间的比例与第一比例一致;The first determining unit is configured to determine second relative position information of multiple target points in real space according to the first ratio between the first distance and the actual space distance and the first relative position information, wherein the first relative position The ratio between the information and the second relative position information is consistent with the first ratio;
第二确定单元,用于根据第二相对位置信息确定世界坐标。The second determining unit is configured to determine the world coordinates according to the second relative position information.
在本申请的一个或多个实施例中,拍摄对位装置300还可以包括:In one or more embodiments of the present application, the
处理模块,用于对参考图片和第一图片进行特征检测处理,得到参考图片到第一图片的转换矩阵;A processing module, configured to perform feature detection processing on the reference picture and the first picture, to obtain a conversion matrix from the reference picture to the first picture;
透视变换模块,用于根据转换矩阵以及第一坐标,进行透视变换,得到第二坐标;The perspective transformation module is used to perform perspective transformation according to the conversion matrix and the first coordinates to obtain the second coordinates;
第二计算模块,用于根据预设的相机外参计算算法,对世界坐标、第二坐标、第一电子设备的内参信息以及第一电子设备的相机畸变系数进行计算,得到第二相机外参信息。The second calculation module is configured to calculate the world coordinates, the second coordinates, the internal parameter information of the first electronic device, and the camera distortion coefficient of the first electronic device according to a preset camera external parameter calculation algorithm to obtain the second camera external parameters information.
在本申请的一个或多个实施例中,第一相机外参信息以及第二相机外参信息均通过旋转向量的形式进行表示;第一确定模块306包括:In one or more embodiments of the present application, both the first camera extrinsic information and the second camera extrinsic information are expressed in the form of rotation vectors; the first determining
转换单元,用于将第一相机外参信息以及第二相机外参信息分别转换为旋转矩阵,得到第一相机外参信息对应的第一旋转矩阵,以及第二相机外参信息对应的第二旋转矩阵;The conversion unit is configured to convert the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain the first rotation matrix corresponding to the first camera extrinsic information and the second rotation matrix corresponding to the second camera extrinsic information. rotation matrix;
第三确定单元,用于根据第一旋转矩阵和第二旋转矩阵,确定机械臂的运动控制参数。The third determining unit is configured to determine motion control parameters of the mechanical arm according to the first rotation matrix and the second rotation matrix.
在本申请的一个或多个实施例中,运动控制参数包括机械臂的偏移信息以及姿态调整角度;第三确定单元用于:In one or more embodiments of the present application, the motion control parameters include the offset information of the robotic arm and the attitude adjustment angle; the third determining unit is used for:
将第一旋转矩阵与第二旋转矩阵的转置矩阵相乘,得到姿态调整角度,以及将第一旋转矩阵与第二旋转矩阵进行相减,得到偏移信息。The first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and the first rotation matrix is subtracted from the second rotation matrix to obtain offset information.
需要说明的是,本申请实施例提供的拍摄对位方法,执行主体可以为拍摄对位装置,或者拍摄对位装置中的用于执行拍摄对位方法的控制模块。本申请实施例中以拍摄对位装置执行拍摄对位方法为例,说明本申请实施例提供的拍摄对位装置。It should be noted that, the photographing alignment method provided in the embodiment of the present application may be executed by a photographing alignment device, or a control module in the photographing alignment device for executing the photographing alignment method. In the embodiment of the present application, the photographing alignment device provided by the embodiment of the present application is described by taking the photographing alignment device executing the photographing alignment method as an example.
本申请实施例中的拍摄对位装置可以是装置,也可以是终端中的部件、集成电路、或芯片。该装置可以是移动电子设备,也可以为非移动电子设备。示例性的,移动电子设备可以为手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、可穿戴设备、超级移动个人计算机(ultra-mobile personal computer,UMPC)、上网本或者个人数字助理(personaldigital assistant,PDA)等,非移动电子设备可以为服务器、网络附属存储器(NetworkAttached Storage,NAS)、个人计算机(personal computer,PC)、电视机(television,TV)、柜员机或者自助机等,本申请实施例不作具体限定。The photographing alignment device in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant). , PDA), etc., non-mobile electronic equipment can be server, network attached storage (NetworkAttached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., the embodiment of the present application Not specifically limited.
本申请实施例中的拍摄对位装置可以为具有操作系统的装置。该操作系统可以为安卓(Android)操作系统,可以为ios操作系统,还可以为其他可能的操作系统,本申请实施例不作具体限定。The shooting alignment device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.
本申请实施例提供的拍摄对位装置能够实现图2或图11的方法实施例实现的各个过程,为避免重复,这里不再赘述。The photographing alignment device provided in the embodiment of the present application can realize various processes realized by the method embodiment in FIG. 2 or FIG. 11 , and details are not repeated here to avoid repetition.
本申请还提供一种电子设备,包括处理器,存储器及存储在存储器上并可在处理器上运行的程序或指令,程序或指令被处理器执行时实现上述任意一项实施例的拍摄对位方法的步骤。The present application also provides an electronic device, including a processor, a memory, and a program or instruction stored on the memory and operable on the processor. When the program or instruction is executed by the processor, the shooting alignment of any of the above-mentioned embodiments can be realized. method steps.
如图13所示,本申请实施例还提供一种控制设备400,包括处理器401,存储器402,存储在存储器402上并可在处理器401上运行的程序或指令,该程序或指令被处理器401执行时实现上述拍摄对位方法实施例的各个过程,且能达到相同的技术效果,为避免重复,这里不再赘述。As shown in FIG. 13, the embodiment of the present application also provides a
图13是本申请提供的一种控制设备的另一个实施例的结构示意图。Fig. 13 is a schematic structural diagram of another embodiment of a control device provided by the present application.
控制设备500包括但不限于:射频单元501、网络模块502、音频输出单元503、输入单元504、传感器505、显示单元506、用户输入单元507、接口单元508、存储器509、以及处理器510等部件。The
本领域技术人员可以理解,控制设备500还可以包括给各个部件供电的电源(比如电池),电源可以通过电源管理系统与处理器510逻辑相连,从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。图14中示出的设备结构并不构成对控制设备的限定,控制设备可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置,在此不再赘述。Those skilled in the art can understand that the
其中,处理器510用于:Wherein, the
获取第一图片以及预存储的参考图片,第一图片为第一电子设备被机械臂控制在第一位姿下拍摄的图片;Acquiring a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device controlled by the mechanical arm in the first position;
获取在拍摄参考图片时的第一相机外参信息,以及获取在拍摄第一图片时的第二相机外参信息;Obtaining the extrinsic information of the first camera when taking the reference picture, and acquiring the extrinsic information of the second camera when taking the first picture;
根据第一相机外参信息以及第二相机外参信息,确定机械臂的运动控制参数;Determining motion control parameters of the robotic arm according to the external parameter information of the first camera and the external parameter information of the second camera;
控制机械臂按照运动控制参数进行运动,其中,第一电子设备随着机械臂从第一位姿运动至第二位姿,以使第一电子设备在第二位姿拍摄出与参考图片的构图匹配的第二图片。Controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the robotic arm, so that the first electronic device takes a composition with the reference picture in the second pose Match the second picture.
本申请实施例中的拍摄对位是依据预先存储的参考图片和实际拍摄的第一图片完成的,参考图片内的物体和场景可以进行替换,对固定场景无固定要求。因此,如果需要对第一电子设备在多个场景(比如近拍和远拍)的拍摄效果进行测试,则提供对各个场景拍摄出的参考图片,即可完成对应场景的拍摄效果测试,让多个场景的快速切换测试成为可能。另外,本申请实施例可以适用于不同相机的拍摄效果测试。如此,满足了多场景和多相机的需求。而且,本申请实施例针对不固定被拍摄物体,可适用于任意物体的拍照位姿对位,所拍摄图片可以较大程度上满足预想的参考图片的构图。The shooting alignment in the embodiment of the present application is completed based on the pre-stored reference picture and the first picture actually taken. Objects and scenes in the reference picture can be replaced, and there is no fixed requirement for fixed scenes. Therefore, if it is necessary to test the shooting effect of the first electronic device in multiple scenes (such as close-up shooting and long-distance shooting), the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. It is possible to quickly switch between different scenarios for testing. In addition, the embodiment of the present application may be applicable to shooting effect tests of different cameras. In this way, the requirements of multiple scenes and multiple cameras are met. Moreover, the embodiment of the present application is aimed at unfixed objects to be photographed, and is applicable to photographing pose alignment of any object, and the photographed pictures can satisfy the expected composition of the reference picture to a large extent.
在本申请的一个或多个实施例中,第一相机外参信息表征世界坐标到第一坐标的转换关系,第二相机外参信息表征世界坐标到第二坐标的转换关系,其中,世界坐标为参考图片上的多个目标点在世界坐标系中的坐标,第一坐标为多个目标点在参考图片上的坐标,第二坐标为多个目标点在第一图片上的坐标。In one or more embodiments of the present application, the first camera extrinsic information represents the transformation relationship from the world coordinate to the first coordinate, and the second camera extrinsic information represents the transformation relation from the world coordinate to the second coordinate, wherein the world coordinate are the coordinates of multiple target points on the reference picture in the world coordinate system, the first coordinates are the coordinates of the multiple target points on the reference picture, and the second coordinates are the coordinates of the multiple target points on the first picture.
在本申请的一个或多个实施例中,处理器510具体用于:In one or more embodiments of the present application, the
获取两个定位标在参考图片上的第一距离;Obtain the first distance between the two positioning markers on the reference picture;
根据第一距离以及两个定位标的实际空间距离,确定多个目标点在世界坐标系中的世界坐标;Determine the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual space distance between the two positioning marks;
根据预设的相机外参计算算法,对世界坐标、第一坐标、拍摄参考图片的第二电子设备的内参信息以及第二电子设备的相机畸变系数进行计算,得到第一相机外参信息。According to the preset camera extrinsic parameter calculation algorithm, the world coordinates, the first coordinates, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device are calculated to obtain the first camera extrinsic information.
在本申请的一个或多个实施例中,处理器510具体用于:In one or more embodiments of the present application, the
获取多个目标点在参考图片上的第一相对位置信息;Obtaining first relative position information of multiple target points on the reference picture;
根据第一距离与实际空间距离之间的第一比例以及第一相对位置信息,确定多个目标点在实际空间中的第二相对位置信息,其中,第一相对位置信息与第二相对位置信息之间的比例与第一比例一致;According to the first ratio between the first distance and the actual space distance and the first relative position information, determine the second relative position information of multiple target points in the actual space, wherein the first relative position information and the second relative position information The ratio between is consistent with the first ratio;
根据第二相对位置信息确定世界坐标。The world coordinates are determined according to the second relative position information.
在本申请的一个或多个实施例中,处理器510具体用于:In one or more embodiments of the present application, the
对参考图片和第一图片进行特征检测处理,得到参考图片到第一图片的转换矩阵;performing feature detection processing on the reference picture and the first picture to obtain a transformation matrix from the reference picture to the first picture;
根据转换矩阵以及第一坐标,进行透视变换,得到第二坐标;Perform perspective transformation according to the transformation matrix and the first coordinate to obtain the second coordinate;
根据预设的相机外参计算算法,对世界坐标、第二坐标、第一电子设备的内参信息以及第一电子设备的相机畸变系数进行计算,得到第二相机外参信息。The world coordinates, the second coordinates, the internal parameter information of the first electronic device, and the camera distortion coefficient of the first electronic device are calculated according to the preset camera external parameter calculation algorithm to obtain the second camera external parameter information.
在本申请的一个或多个实施例中,第一相机外参信息以及第二相机外参信息均通过旋转向量的形式进行表示;处理器510具体用于:In one or more embodiments of the present application, both the first camera extrinsic parameter information and the second camera extrinsic parameter information are represented in the form of rotation vectors; the
将第一相机外参信息以及第二相机外参信息分别转换为旋转矩阵,得到第一相机外参信息对应的第一旋转矩阵,以及第二相机外参信息对应的第二旋转矩阵;converting the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain a first rotation matrix corresponding to the first camera extrinsic information and a second rotation matrix corresponding to the second camera extrinsic information;
根据第一旋转矩阵和第二旋转矩阵,确定机械臂的运动控制参数。According to the first rotation matrix and the second rotation matrix, the motion control parameters of the mechanical arm are determined.
在本申请的一个或多个实施例中,处理器510具体用于:In one or more embodiments of the present application, the
将第一旋转矩阵与第二旋转矩阵的转置矩阵相乘,得到姿态调整角度,以及将第一旋转矩阵与第二旋转矩阵进行相减,得到偏移信息。The first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and the first rotation matrix is subtracted from the second rotation matrix to obtain offset information.
应理解的是,本申请实施例中,输入单元504可以包括图形处理器(GraphicsProcessing Unit,GPU)5041和麦克风5042,图形处理器5041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。显示单元506可包括显示面板5061,可以采用液晶显示器、有机发光二极管等形式来配置显示面板5061。用户输入单元507包括触控面板5071以及其他输入设备5072。触控面板5071,也称为触摸屏。触控面板5071可包括触摸检测装置和触摸控制器两个部分。其他输入设备5072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆,在此不再赘述。存储器509可用于存储软件程序以及各种数据,包括但不限于应用程序和操作系统。处理器510可集成应用处理器和调制解调处理器,其中,应用处理器主要处理操作系统、用户界面和应用程序等,调制解调处理器主要处理无线通信。可以理解的是,上述调制解调处理器也可以不集成到处理器510中。It should be understood that, in the embodiment of the present application, the
本申请实施例还提供一种可读存储介质,所述可读存储介质上存储有程序或指令,该程序或指令被处理器执行时实现上述拍摄对位方法实施例的各个过程,且能达到相同的技术效果,为避免重复,这里不再赘述。The embodiment of the present application also provides a readable storage medium, the readable storage medium stores a program or an instruction, and when the program or instruction is executed by the processor, the various processes of the above-mentioned embodiment of the shooting alignment method are realized, and can achieve The same technical effects are not repeated here to avoid repetition.
其中,所述处理器为上述实施例中所述的控制设备中的处理器。所述可读存储介质,包括计算机可读存储介质,如计算机只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等。Wherein, the processor is the processor in the control device described in the above embodiments. The readable storage medium includes computer readable storage medium, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.
本申请实施例另提供了一种芯片,所述芯片包括处理器和通信接口,所述通信接口和所述处理器耦合,所述处理器用于运行程序或指令,实现上述拍摄对位方法实施例的各个过程,且能达到相同的技术效果,为避免重复,这里不再赘述。The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to realize the above-mentioned embodiment of the shooting alignment method Each process, and can achieve the same technical effect, in order to avoid repetition, will not repeat them here.
应理解,本申请实施例提到的芯片还可以称为系统级芯片、系统芯片、芯片系统或片上系统芯片等。It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.
需要说明的是,在本文中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外,需要指出的是,本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能,还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能,例如,可以按不同于所描述的次序来执行所描述的方法,并且还可以添加、省去、或组合各种步骤。另外,参照某些示例所描述的特征可在其他示例中被组合。It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以计算机软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端(可以是手机,计算机,服务器,或者网络设备等)执行本申请各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.
上面结合附图对本申请的实施例进行了描述,但是本申请并不局限于上述的具体实施方式,上述的具体实施方式仅仅是示意性的,而不是限制性的,本领域的普通技术人员在本申请的启示下,在不脱离本申请宗旨和权利要求所保护的范围情况下,还可做出很多形式,均属于本申请的保护之内。The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.
Claims (11)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110599186.7A CN113329179B (en) | 2021-05-31 | 2021-05-31 | Shooting alignment method, device, equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110599186.7A CN113329179B (en) | 2021-05-31 | 2021-05-31 | Shooting alignment method, device, equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113329179A CN113329179A (en) | 2021-08-31 |
CN113329179B true CN113329179B (en) | 2023-04-07 |
Family
ID=77422685
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110599186.7A Active CN113329179B (en) | 2021-05-31 | 2021-05-31 | Shooting alignment method, device, equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113329179B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113506299B (en) * | 2021-09-13 | 2021-12-10 | 武汉逸飞激光股份有限公司 | Soft-package battery cell feeding control method and device, electronic equipment and storage medium |
CN114004890B (en) * | 2021-11-04 | 2023-03-24 | 如你所视(北京)科技有限公司 | Attitude determination method and apparatus, electronic device, and storage medium |
CN114449165B (en) * | 2021-12-27 | 2023-07-18 | 广州极飞科技股份有限公司 | Photographing control method and device, unmanned equipment and storage medium |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112837381A (en) * | 2021-02-09 | 2021-05-25 | 上海振华重工(集团)股份有限公司 | Camera calibration method, system and equipment suitable for driving equipment |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108347559A (en) * | 2018-01-05 | 2018-07-31 | 深圳市金立通信设备有限公司 | A kind of image pickup method, terminal and computer readable storage medium |
CN110933297B (en) * | 2019-11-12 | 2021-11-23 | 武汉联一合立技术有限公司 | Photographing control method and device of intelligent photographing system, storage medium and system |
WO2021102914A1 (en) * | 2019-11-29 | 2021-06-03 | 深圳市大疆创新科技有限公司 | Trajectory repeating method and system, movable platform and storage medium |
CN111654624B (en) * | 2020-05-29 | 2021-12-24 | 维沃移动通信有限公司 | Shooting prompting method and device and electronic equipment |
CN112489113B (en) * | 2020-11-25 | 2024-06-11 | 深圳地平线机器人科技有限公司 | Camera external parameter calibration method and device and camera external parameter calibration system |
-
2021
- 2021-05-31 CN CN202110599186.7A patent/CN113329179B/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112837381A (en) * | 2021-02-09 | 2021-05-25 | 上海振华重工(集团)股份有限公司 | Camera calibration method, system and equipment suitable for driving equipment |
Also Published As
Publication number | Publication date |
---|---|
CN113329179A (en) | 2021-08-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110809786B (en) | Calibration device, calibration chart, chart pattern generation device, and calibration method | |
CN113329179B (en) | Shooting alignment method, device, equipment and storage medium | |
CN109313799B (en) | Image processing method and apparatus | |
US7554575B2 (en) | Fast imaging system calibration | |
CN113409391B (en) | Visual positioning method and related device, equipment and storage medium | |
CN108288294A (en) | A kind of outer ginseng scaling method of a 3D phases group of planes | |
CN113841384B (en) | Calibration device, chart for calibration and calibration method | |
CN108765498A (en) | Monocular vision tracking, device and storage medium | |
CN110807814A (en) | Camera pose calculation method, device, equipment and storage medium | |
CN107578450B (en) | Method and system for calibrating assembly error of panoramic camera | |
CN113838151B (en) | Camera calibration method, device, equipment and medium | |
CN113393505A (en) | Image registration method, visual positioning method, related device and equipment | |
CN117173254A (en) | Camera calibration method, system, device and electronic equipment | |
CN119131335B (en) | Target positioning method, device, equipment and storage medium | |
JP6304815B2 (en) | Image processing apparatus and image feature detection method, program and apparatus thereof | |
CN114463437A (en) | A camera calibration method, apparatus, device and computer readable medium | |
CN110728714B (en) | Image processing method and device, storage medium and electronic equipment | |
CN112446928B (en) | External parameter determining system and method for shooting device | |
CN116862992A (en) | Method and device for determining visual field of monitoring camera | |
CN102110292B (en) | Zoom lens calibration method and device in virtual sports | |
CN113379843A (en) | Parameter calibration method, device and equipment for imaging system of camera module | |
CN119478049B (en) | A method, system and related device for stitching images taken by multiple cameras for appearance inspection | |
CN115278071B (en) | Image processing method, device, electronic equipment and readable storage medium | |
Han et al. | The Calibration of Kinect Camera Based on ARtoolkit | |
CN119942158A (en) | Dual-camera calibration method, system and related device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |