RU2792946C1 - Method and computer device for lidar system calibration - Google Patents

Abstract

FIELD: lidar systems.

SUBSTANCE: present technology relates to methods and electronic devices for driving a vehicle, and more particularly to methods and electronic devices for calibrating a lidar system. Methods and devices for determining the axis of symmetry of an unmanned vehicle (SDV) and for calibrating a lidar (LIDAR) optical detection and ranging system are disclosed. One of the coordinate system axes of the lidar system runs along the direction of the normal to the earth's surface. The method includes obtaining a subset of the detected points in the coordinate system; generating a subset of mirror points based on the subset of detected points; projecting the subset of mirror points onto the subset of detected points so as to define pairs of overlapping data points; using symmetrically opposite detected points to determine the SDV symmetry axis in the lidar system coordinate system; and calibrating the lidar system using an angular displacement between the SDV symmetry axis and another of the lidar system coordinate system axes.

EFFECT: improved calibration accuracy of the lidar system.

45 cl, 3 dwg

Description

The technical field to which the invention belongs

[1] The present technology relates to methods and electronic devices for driving a vehicle, and more specifically, to methods and electronic devices for calibrating a lidar system.

State of the art

[2] In the prior art, several computer navigation systems have been proposed and implemented that are capable of assisting in navigation and/or driving. These systems range from more basic map-based localization solutions (i.e. using a computer system to assist the driver in navigating a route from a starting point to a destination) to more complex solutions (computerized and/or driver-autonomous). driving systems).

[3] Some of these systems are implemented as what is commonly known as "optimal speed control" or "cruise control" systems. In these systems, the computer system on board the vehicles maintains a user-specified vehicle speed. Part of the optimal speed control system implements an "intelligent distance control" system whereby a user can set a distance to a potential vehicle ahead (eg, select a value expressed in terms of number of vehicles) and the computer system adjusts the vehicle's speed at least partially by basis of a vehicle approaching a potential vehicle ahead within a predetermined distance. Some speed control systems are additionally equipped with collision management systems, and these systems, when a vehicle (or other obstacle) is detected in front of a moving vehicle, slow or stop the vehicle.

[4] Some advanced systems provide fully autonomous driving of a vehicle without the direct control of an operator (i.e. driver). These autonomous driving-enabled vehicles include computer systems that can prompt the vehicle to accelerate, brake, stop, change lanes, and park automatically.

[5] One of the main technical problems in the implementation of the above computer systems is the ability of the computer system to detect an object located in the vicinity of the vehicle. In one example, computer systems may require the ability to detect a vehicle ahead of the vehicle in question (where the vehicle in question has a computer system on board), where the vehicle in front may pose a risk/danger to the vehicle in question and may require the computer system to take corrective action. measures, whether it be braking or otherwise changing speed, stopping or changing lanes.

[6] Other technical problems with the implementation of the above computer systems are the decalibration of sensors and other systems that collect data regarding the surroundings of the vehicle. Many factors, including weather, road conditions, driving habits, for example, affect sensors and other systems over time that require calibration to ensure that data is accurately collected and used correctly by other vehicle control systems. .

The essence of the invention

[7] The developers of the present technology have found that the solution of the prior art has disadvantages.

[8] In a first broad aspect of the present technology, a method for calibrating a lidar (LIDAR) optical detection and ranging system is provided. The lidar system has a coordinate system. The coordinate system has axes. One of the axes runs along the direction normal to the earth's surface. The lidar system can be mounted on an unmanned vehicle (SDV). SDV is located on the surface of the earth. The method is carried out by means of an electronic device associated with the SDV. The method comprises receiving, by means of an electronic device, from the lidar system a subset of detected data points in the coordinate system of the lidar system and an image of the outer surface of the SDV. The subset of detected data points is a subset of the set of detected data points of the lidar system. The method comprises generating, by the electronic device, a subset of mirror image data points based on the subset of detected data points. A subset of mirror image data points represents a mirror image of the outer surface of the SDV. The method comprises projecting, by means of an electronic device, a subset of mirror image data points onto a subset of detected data points so as to define pairs of overlapping data points. Each given pair of overlapping data points includes: (i) a first data point from a subset of detected data points, and (ii) a second data point from a subset of mirror image data points such that the second data point is a mirror image of the third data point in the subset detected data points, which is located symmetrically opposite the first data point. The method comprises using, via an electronic device, (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points, to determine the axis of symmetry of the SDV in the system coordinates of the lidar system. The method comprises calibrating, by means of an electronic device, the lidar system using an angular displacement between the SDV symmetry axis and another of the coordinate system axes of the lidar system.

[9] In some embodiments of the method, the method further comprises, prior to obtaining a subset of detected data points, obtaining a plurality of detected data points from the lidar system.

[10] In some embodiments of the method, the method further comprises filtering, by the electronic device, the plurality of detected data points to determine a subset of the detected data points.

[11] In some embodiments of the method, filtering the plurality of detected data points comprises applying, by means of the electronic device, an approximation rule to the plurality of detected data points detected by the lidar system.

[12] In some embodiments of the method, the approximation rule comprises an indication of a threshold distance that is considered to include the detected subset of data points.

[13] In some embodiments of the method, prior to obtaining a plurality of detected data points, the method further comprises causing the electronic indication device to increase the outer surface of the SDV.

[14] In some embodiments of the method, providing an indication for an increase in the outer surface comprises providing, by means of an electronic indication device, for the opposite SDV doors to be fully open.

[15] In some embodiments of the method, providing an indication for an increase in the outer surface further comprises providing, by means of an electronic indication device, for the full opening of the trunk of the SDV.

[16] In some embodiments of the method, electronically projecting a subset of mirror image data points onto a subset of detected data points comprises electronically iteratively minimizing the distance between the subset of mirror image data points and the subset of detected data points.

[17] In some embodiments of the method, iterative distance minimization comprises defining a transformation rule between a subset of mirror image data points and a subset of detected data points so as to overlap a given mirror image data point with a given detected data point. This mirror image data point is formed based on this other detected data point, which is located symmetrically opposite this detected data point.

[18] In some embodiments of the method, iterative distance minimization comprises electronically applying an iterative nearest point (ICP) algorithm.

[19] In some embodiments of the method, (i) at least two first data points and (ii) at least two corresponding third data points comprise, respectively, two first data points and (ii) two corresponding third data points.

[20] In some embodiments of the method, (i) at least two first data points and (ii) at least two corresponding third data points comprise, respectively, three first data points and (ii) three corresponding third data points.

[21] In some embodiments of the method, using (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points to determine the axis of symmetry of the SDV, contains:

- selecting by means of an electronic device (i) at least two first data points and (ii) at least two corresponding third data points;

- determining by means of an electronic device a midpoint for each pair of (i) at least two first data points and (ii) at least two corresponding third data points; And

- determination by means of an electronic device of a line through the respective midpoints as the axis of symmetry of the SDV.

[22] In some embodiments of the method, the coordinate system of the lidar system has three orthogonal axes.

[23] In some embodiments of the method, a plurality of detected data points are collected in a spherical coordinate system and converted to a lidar coordinate system having three orthogonal axes.

[24] In some embodiments of the method, a subset of mirror image data points is formed in the coordinate system of the lidar system.

[25] In some embodiments of the method, a subset of mirror image data points is generated in a multidimensional space identical to the multidimensional space of a subset of detected data points.

[26] In some embodiments, prior to calibration, the method further comprises: (i) electronically determining the direction of a normal to the earth's surface based on a plurality of detected lidar system data points, and (ii) electronically tying one of the axes of the coordinate system based on the direction of the normal to the ground surface in such a way that one of the axes runs in the direction of the normal.

[27] In some embodiments, the method further comprises electronically determining an angular displacement between an axis of symmetry of the SDV and one of the coordinate system axes of the lidar system.

[28] In some embodiments of the method, using the angular displacement comprises: (i) electronically determining a rotational displacement rule based on the angular displacement such that another of the lidar system coordinate system axes coincides with the SDV axis of symmetry, and (ii) using rotational shift rule electronics for rotating the usable detected data points of the lidar system, thereby determining externally calibrated usable detected data points.

[29] In some embodiments, the method further comprises iteratively repeating the calibration of the lidar system during operation of the SDV.

[30] In a second broad aspect of the present technology, a method for determining the axis of symmetry of an unmanned vehicle (SDV) is provided. The SDV can be equipped with a lidar (LIDAR) optical detection and ranging system. The lidar system has a coordinate system. The coordinate system has axes, the method is carried out by means of an electronic device associated with the SDV. The method comprises receiving, by means of an electronic device, from the lidar system a subset of detected data points in the coordinate system of the lidar system and an image of the outer surface of the SDV. The subset of detected data points is a subset of the set of detected data points of the lidar system. The method comprises generating, by the electronic device, a subset of mirror image data points based on the subset of detected data points. A subset of mirror image data points represents a mirror image of the outer surface of the SDV. The method comprises projecting, by means of an electronic device, a subset of mirror image data points onto a subset of detected data points so as to define pairs of overlapping data points. Each given pair of overlapping data points includes: (i) a first data point from a subset of detected data points, and (ii) a second data point from a subset of mirror image data points such that the second data point is a mirror image of the third data point in the subset detected data points, which is located symmetrically opposite the first data point. The method comprises using, via an electronic device, (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points, to determine the axis of symmetry of the SDV in the system coordinates of the lidar system.

[31] In a third broad aspect of the present technology, an electronic device is provided for calibrating a (lidar) optical detection and ranging system. The lidar system has a coordinate system. The coordinate system has axes. One of the axes runs along the direction normal to the earth's surface. The lidar system can be mounted on an unmanned vehicle (SDV). SDV is located on the surface of the earth. The electronic device is configured to receive from the lidar system a subset of the detected data points in the lidar system coordinate system and represent the outer surface of the SDV. The subset of detected data points is a subset of the set of detected data points of the lidar system. The electronic device is configured to generate a subset of mirror image data points based on the subset of detected data points. A subset of mirror image data points represents a mirror image of the outer surface of the SDV. The electronic device is configured to project a subset of mirror image data points onto a subset of detected data points so as to define pairs of overlapping data points. Each given pair of overlapping data points includes: (i) a first data point from a subset of detected data points, and (ii) a second data point from a subset of mirror image data points such that the second data point is a mirror image of the third data point in the subset detected data points, which is located symmetrically opposite the first data point. The electronic device is configured to use (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points to determine the SDV symmetry axis in the lidar coordinate system. systems. The electronic device is configured to calibrate the lidar system using an angular displacement between the SDV symmetry axis and another of the coordinate system axes of the lidar system.

[32] In some embodiments, the electronic device is further configured to, prior to receiving a subset of detected data points, receive a plurality of detected data points from the lidar system.

[33] In some embodiments, the electronic device is further configured to filter the plurality of detected data points to determine a subset of the detected data points.

[34] In some embodiments, in order for the electronic device to filter the plurality of detected data points, the electronic device is configured to apply the approximation rule to the plurality of detected data points detected by the lidar system.

[35] In some embodiments of the electronic device, the approximation rule contains an indication of a threshold distance, which is considered to include the detected subset of data points.

[36] In some embodiments, the electronic device is further configured to provide an indication to increase the outer surface of the SDV prior to receiving a plurality of detected data points.

[37] In some embodiments of the electronic device, providing an indication of an increase in the outer surface comprises configuring the electronic device to provide indications for the full opening of the opposite SDV doors.

[38] In some embodiments of the electronic device, providing an indication of an increase in the outer surface comprises configuring the electronic device to provide an indication of the full opening of the trunk of the SDV.

[39] In some embodiments, to enable the electronic device to project a subset of mirror image data points onto a subset of detected data points, the electronic device is configured to iteratively minimize the distance between the subset of mirror image data points and the subset of detected data points.

[40] In some embodiments of an electronic device, iterative distance minimization comprises executing the electronic device with the ability to determine a conversion rule between a subset of mirror image data points and a subset of detected data points in such a way as to overlap a given mirror image data point with a given detected data point. This mirror image data point is formed based on this other detected data point, which is located symmetrically opposite this detected data point.

[41] In some embodiments of an electronic device, iterative distance minimization comprises executing the electronic device with the ability to apply an iterative closest point (ICP) algorithm.

[42] In some embodiments of the electronic device, (i) at least two first data points and (ii) at least two respective third data points comprise two first data points, respectively, and (ii) two respective third data points.

[43] In some embodiments of the electronic device, (i) at least two first data points and (ii) at least two respective third data points comprise three first data points, respectively, and (ii) three respective third data points.

[44] In some embodiments of an electronic device, the use of (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite the at least two first data points to define the axis of symmetry of the SDV , contains the implementation of an electronic device with the ability to:

- select (i) at least two first data points and (ii) at least two corresponding third data points;

- for each pair of (i) at least two first data points and (ii) at least two corresponding third data points, determine the middle point; And

- define the line through the corresponding midpoints as the axis of symmetry of the SDV.

[45] In some embodiments of the electronic device, the coordinate system of the lidar system has three orthogonal axes.

[46] In some embodiments of the electronic device, a plurality of detected data points are collected in a spherical coordinate system and converted to a lidar coordinate system having three orthogonal axes.

[47] In some embodiments of the electronic device, a subset of mirror image data points is formed in the coordinate system of the lidar system.

[48] In some embodiments of the electronic device, a subset of mirror image data points is formed in a multidimensional space identical to the multidimensional space of a subset of detected data points.

[49] In some embodiments of the electronic device, the electronic device is further configured prior to calibration to: (i) determine the direction of a normal to the earth's surface based on a plurality of detected lidar system data points, and (ii) tie one of the axes of the coordinate system based on the direction of the normal to the earth's surface. ground surface in such a way that one of the axes runs in the direction of the normal.

[50] In some embodiments, the implementation of the electronic device is further configured to determine the angular displacement between the axis of symmetry SDV and one of the axes of the coordinate system of the lidar system.

[51] In some embodiments of an electronic device, using angular displacement comprises configuring the electronic device to (i) determine a rotational displacement rule based on the angular displacement such that another of the lidar system coordinate system axes coincides with the SDV axis of symmetry, and (ii) use a rotational shift rule to rotate the usable detected data points of the lidar system, thereby defining externally calibrated usable detected data points.

[52] In some embodiments, the electronic device is further configured to iteratively calibrate the lidar system during operation of the SDV.

[53] In a fourth broad aspect of the present technology, an electronic device is provided for determining the axis of symmetry of an unmanned vehicle (SDV), wherein an optical detection and ranging (lidar) system can be installed in the SDV. The lidar system has a coordinate system. The coordinate system has axes. SDV is located on the surface of the earth. The electronic device is configured to receive from the lidar system a subset of the detected data points in the lidar system coordinate system and represent the outer surface of the SDV. The subset of detected data points is a subset of the set of detected data points of the lidar system. The electronic device is configured to generate a subset of mirror image data points based on the subset of detected data points. A subset of mirror image data points represents a mirror image of the outer surface of the SDV. The electronic device is configured to project a subset of mirror image data points onto a subset of detected data points so as to define pairs of overlapping data points. Each given pair of overlapping data points includes: (i) a first data point from a subset of detected data points, and (ii) a second data point from a subset of mirror image data points such that the second data point is a mirror image of the third data point in the subset detected data points, which is located symmetrically opposite the first data point. The electronic device is configured to use (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points to determine the SDV symmetry axis in the lidar coordinate system. systems.

[54] As used herein, a "server" is a computer program that runs on appropriate hardware and is capable of receiving requests (eg, from client devices) over a network and making those requests or causing those requests to be made. The hardware may be implemented as one physical computer or one physical computer system, but neither is necessarily the case with respect to the present technology. In the present context, the use of the term "server" does not imply that every task (e.g., instructions or requests received) or any particular task must be received, executed, or caused to be executed by an identical server (i.e., identical software and/or hardware). ); it means that any number of software elements or hardware devices can be provided when receiving/sending, performing or inducing to perform any task or request, or the results of any task or request; and all such software and hardware may be a single server or multiple servers, all of which is included in the expression "at least one server".

[55] In the context of the present specification, an "electronic device" is any computer hardware that is capable of executing software that is appropriate for the relevant task in question. In the context of the present description of the invention, the term "electronic device" means that the device can function as a server for other electronic devices and client devices; however, this need not necessarily be the case with respect to the present technology. Thus, some (non-limiting) examples of electronic devices include personal computers (desktops, laptops, netbooks, etc.), smartphones and tablets, and network equipment such as routers, switches, and gateways. It should be understood that in the present context, the fact that a device functions as an electronic device does not mean that it cannot function as a server for other electronic devices. The use of the term "electronic device" does not preclude the use of multiple client devices in receiving/sending, performing or causing any task or request, or the results of any task or request, or steps in any method described herein.

[56] In the context of the present specification, a "client device" is any computer hardware that is capable of executing software corresponding to the relevant task in question. In the context of the present description of the invention, the term "client device" is generally associated with the user of the client device. Thus, some (non-limiting) examples of client devices include personal computers (desktops, laptops, netbooks, etc.), smartphones and tablets, and network equipment such as routers, switches, and gateways. It should be noted that a device serving as a client device in the present context is not excluded from serving as a server for other client devices. The use of the term "client device" does not preclude the use of multiple client devices in receiving/sending, performing or causing any task or request, or the results of any task or request, or steps in any method described herein.

[57] In the context of the present description of the invention, the expression "information" includes information in general of any nature or kind, capable of being stored in a database. Thus, information includes, but is not limited to, audiovisual works (images, films, sound recordings, presentations, etc.), data (location data, numerical data, etc.), text (opinions, comments, questions , messages, etc.), documents, spreadsheets, etc.

[58] In the context of the present description of the invention, the expression "software component" includes software (corresponding to a particular hardware context), which is necessary and sufficient in order to implement the specific referenced function(s).

[59] In the context of the present description of the invention, the expression "computer storage media" (also referred to as "data storage media") includes media in general of any nature and form, including, without limitation, RAM, ROM, disks (CD-ROM , DVD, floppy disks, hard disk drives, etc.), flash drives, semiconductor drives, tape drives, etc. A plurality of components may be combined to form computer storage media including two or more media components of the same type and/or two or more media components of different types.

[60] As used herein, a "database" is any structured collection of data, regardless of its particular structure, database management software, or computer hardware on which the data is stored, implemented, or otherwise made available for use. The database may reside on identical hardware with a process that stores or uses the information stored in the database, or it may reside on separate hardware such as a dedicated server or multiple servers.

[61] In the context of the present description of the invention, the words "first", "second", "third", etc. are used as adjectives only for the purpose of distinguishing the nouns they modify from each other, and not for the purpose of describing specific relationships between those nouns. Thus, for example, it should be understood that the use of the terms "first database" and "third server" does not imply a particular order, type, chronology, hierarchy, or ranking (for example) relative to the server, and their use does not (directly) imply that " second server" must necessarily exist in any given situation. Additionally, as explained herein in other contexts, reference to a "first" element and a "second" element does not preclude that the two elements represent an identical actual real world element. Thus, for example, in some cases, the "first" server and the "second" server may be identical software and/or hardware components, in other cases they may be different software and/or hardware components.

[62] Implementations of the present technology have at least one of the above objectives and/or aspects, but do not necessarily have all of them. It should be understood that some aspects of the present technology, which are obtained as a result of an attempt to achieve the above goal, may not satisfy this goal and/or may satisfy other goals not specifically set forth in this document.

[63] Additional and/or alternative features, aspects, and advantages of implementations of the present technology should become apparent from the description below, the accompanying drawings, and the appended claims.

Brief description of the drawings

[64] These and other features, aspects and advantages of the present technology should become more apparent from the following description, the appended claims and the accompanying drawings, in which:

[65] FIG. 1 illustrates a schematic diagram of an exemplary computer system for implementing specific embodiments of systems and/or methods of the present technology.

[66] FIG. 2 illustrates a networked computing environment suitable for use with some embodiments of the present technology.

[67] FIG. 3 illustrates a bird's eye view of the networked computing environment vehicle of FIG. 2 in a non-limiting embodiment of the present technology.

[68] FIG. 4 illustrates a bird's-eye view of a plurality of detected data points in multi-dimensional space provided by the lidar network computing environment system of FIG. 2 in a non-limiting embodiment of the present technology.

[69] FIG. 5 illustrates a schematic illustration of a plurality of computerized operations that the network computing environment electronic device of FIG. 2 is configured to perform to determine the axis of symmetry of the vehicle of the networked computing environment of FIG. 2 in at least some embodiments of the present technology.

[70] FIG. 6 illustrates a subset of detected data points provided by the lidar network computing environment system of FIG. 2 representing the outer surface of the vehicle of the networked computing environment of FIG. 2 and a subset of mirror image data points generated by the networked environment electronics of FIG. 2 showing a mirror image of the outer surface of the vehicle of the networked computing environment of FIG. 2 in a non-limiting embodiment of the present technology.

[71] FIG. 7 illustrates a plurality of overlapping data points defined by the networked environment electronics of FIG. 2 as a result of the projection operation of FIG. 5 in a non-limiting embodiment of the present technology.

[72] FIG. 8 illustrates the axis of symmetry of the vehicle of the networked computing environment of FIG. 2 as determined by the network computing environment electronic device of FIG. 2 in at least some embodiments of the present technology.

[73] FIG. 9 illustrates a flowchart of the method in the networked computing environment system of FIG. 2 in accordance with non-limiting embodiments of the present technology.

[74] FIG. 10 illustrates a schematic illustration of how angular offsets and rotational shift rules are determined by the electronic device of FIG. 2 in accordance with non-limiting embodiments of the present technology.

Detailed description of the invention

[75] The examples and conventions set forth herein are intended primarily to assist the reader in understanding the principles of the present technology, rather than limiting its scope to such specific examples and terms. It should be appreciated that various arrangements may be devised by those skilled in the art which, although not explicitly described or shown herein, nevertheless implement the principles of the present technology and are included within its spirit and scope.

[76] In addition, as an aid to understanding, the following description may describe relatively simplified implementations of the present technology. Those skilled in the art will appreciate that various implementations of the present technology may be highly complex.

[77] In some cases, what are considered useful examples of modifications to the present technology may also be set forth. This is merely as an aid to understanding and, furthermore, should not define the scope or set limits of the present technology. These modifications are not intended to be an exhaustive list, and other modifications may be made by those skilled in the art while staying within the scope of the present technology. Additionally, if examples of modifications are not set forth, this should not be interpreted in such a way that modifications are not possible, and/or in such a way that what is described is the only way to implement this element of the present technology.

[78] In addition, all statements in this document that set forth the principles, aspects and implementations of the technology, as well as their specific examples, cover their design and functional equivalents, regardless of whether they are currently known or developed in the future. Thus, for example, those skilled in the art will recognize that the flowcharts herein represent conceptual views of an illustrative circuit embodying the principles of the present technology. Likewise, it should be appreciated that all flowcharts, block diagrams, state transition diagrams, pseudocode, and the like. represent various processes, which may be represented for the most part on computer-readable media, and are therefore executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[79] The functions of the various elements shown in the drawings, including all functional blocks labeled "processor", can be provided by dedicated hardware, as well as hardware capable of executing software, in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of separate processors, some of which may be shared. In addition, explicit use of the term "processor" or "controller" should not be construed as meaning solely hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, dedicated an integrated circuit (ASIC), a field programmable gate array (FPGA), a read-only memory (ROM) for storing software, a random access memory (RAM), and a non-volatile storage device. Other hardware, conventional and/or specialized, may also be included.

[80] Software modules, or simply modules that are meant to be software, may be represented herein as any combination of flowchart elements, or other elements indicating the execution of process steps, and/or textual description. Such modules may be implemented by hardware, which is shown explicitly or implicitly.

[81] With these basic principles in mind, some non-limiting examples are discussed below to illustrate various implementations of aspects of the present technology.

computer system

[82] Referring initially to FIG. 1, a computer system 100 is shown suitable for use with some implementations of the present technology, the computer system 100 comprising various hardware components including one or more single or multi-core processors collectively represented by a processor 110, a semiconductor storage device 120, a storage device 130 , which can be random access memory or any other type of storage device.

[83] Communication between various components of computer system 100 may be via one or more internal and/or external buses (not shown) (e.g., PCI bus, Universal Serial Bus, IEEE1394 Firewire bus, SCSI bus, Serial ATA bus etc.) to which various hardware components are connected electronically. In accordance with embodiments of the present technology, semiconductor drive 120 stores program instructions suitable for loading into storage device 130 and executed by processor 110 to determine the presence of an object. For example, the program instructions may be part of a vehicle control application executed by processor 110. It should be noted that computer system 100 may have additional and/or optional components (not illustrated), such as network communication modules, localization modules, and the like. .

networked computer environment

[84] FIG. 2 illustrates a networked computing environment 200 suitable for use with some embodiments of systems and/or methods of the present technology. The networked computing environment 200 includes an electronic device 210 associated with a vehicle 220 and/or associated with a user (not illustrated) who can drive the vehicle 220, a server 235 communicating with the electronic device 210 via a communication network 245 (e.g., via the Internet). and the like, as described in more detail later in this document).

[85] Optionally, the networked computing environment 200 may also include a GPS satellite (not illustrated) transmitting and/or receiving a GPS signal to/from the electronic device 210. It should be understood that the present technology is not limited to GPS and may use the technology positioning other than GPS. It should be noted that the GPS satellite can go down at all.

[86] The vehicle 220 with which the electronic device 210 is associated may include any tourist or transportation vehicle, such as a personal or commercial vehicle, truck, motorcycle, or the like. Although vehicle 220 is illustrated as a ground vehicle, this may not be the case in every embodiment of the present technology. For example, vehicle 220 may be a watercraft, such as a boat, or an aircraft, such as an unmanned aerial vehicle.

[87] Vehicle 220 may be a user driven or driverless vehicle. In at least some embodiments of the present technology, it is contemplated that vehicle 220 may be implemented as an unmanned vehicle (SDV). It should be noted that the specific parameters of the vehicle 220 are not limiting, and these specific parameters include: vehicle manufacturer, vehicle model, year of manufacture of the vehicle, vehicle weight, vehicle dimensions, vehicle weight distribution, vehicle surface area vehicle, vehicle height, drive train type (eg 2x or 4x), tire type, brake system, fuel system, mileage, vehicle identification number and engine displacement.

[88] It should be noted that the vehicle 220 can be controlled on the surface 280 of the earth (see Fig. 2). For example, a vehicle 220 may be positioned on a ground surface 280 having a normal direction 285 as illustrated and may move along it during operation.

[89] The implementation of the electronic device 210 is not specifically limited, but as an example, the electronic device 210 may be implemented as a vehicle engine control unit, a vehicle CPU, a vehicle navigation device (e.g., TomTom™, Garmin™), a tablet computer, a personal computer built into the vehicle 220, or the like. Thus, it should be noted that electronic device 210 may or may not be permanently associated with vehicle 220. Additionally or alternatively, electronic device 210 may be implemented in a wireless communication device such as a mobile phone (eg, smartphone or cordless phone). In specific embodiments, the electronic device 210 has a display 270.

[90] The electronic device 210 may include some or all of the components of the computer system 100 illustrated in FIG. 1. In particular embodiments, electronic device 210 is an on-board computer device and includes a processor 110, a semiconductor drive 120, and a storage device 130. In other words, electronic device 210 includes hardware and/or software and/or firmware, or a combination of the foregoing. for data processing, as described in more detail below.

Lidar system

[91] In accordance with non-limiting embodiments of the present technology, the electronic device 210 further comprises or has access to an optical detection and ranging (lidar) system 230 that is generally configured to capture data with respect to at least a portion of the surrounding area 250 of the vehicle. 220.

[92] With reference to FIG. 3, a bird's eye view 300 of a vehicle 220 is illustrated in the surrounding area 250. In this non-limiting implementation, the lidar system 230 is mounted on the vehicle 220 on the roof and the vehicle 220 is located on the ground surface 280.

[93] However, it is contemplated that lidar system 230 may be installed (or upgraded) on vehicle 220 in a variety of locations and/or configurations. For example, the lidar system 230 may be mounted in the cabin, in the upper portion of the windshield of the vehicle 220, but other locations are within the scope of this disclosure, including the rear window, side windows, front hood, roof, grille, front bumper, or side of the vehicle 220. In other non-limiting embodiments of the present technology, the lidar system 230 may be mounted even in a dedicated housing (not illustrated) mounted on top of the vehicle 220.

[94] Regardless of the specific location at which the lidar system 230 is installed on the vehicle 220, as mentioned above, the lidar system 230 is configured to capture data regarding the surroundings 250 of the vehicle 220. The following describes how the lidar system 230 is configured to capture data regarding the surroundings 250 of the vehicle 220.

[95] Generally speaking, lidar system 230 includes, among other things, (i) a radiation emission component and (ii) a radiation detection component. A radiation emitting component of the lidar system 230, such as one or more laser devices (for example), emits radiation to the vicinity of the lidar system 230 (for example, to the surrounding area 250 of the vehicle 220). The emitted radiation is then scattered (and/or, for example, reflected) in multiple directions upon contact with an object in the vicinity of the lidar system 230.

[96] At least a portion of this scattered (and/or reflected) radiation is scattered (and/or reflected) in the direction of the radiation detection component of the lidar system 230, which captures this scattered radiation. After the scattered radiation is captured, the lidar system 230 determines the "time of flight" of the radiation, i. the difference between (i) the time when the radiation is emitted and (ii) the time when the scattered radiation is captured. Using this time of flight and the speed of light, lidar system 230 is able to determine the position of an object that scatters (and/or reflects) the emitted radiation relative to lidar system 230.

[97] The lidar system 230 expresses such a determined position of an object as a detected data point in multidimensional space (eg, three-dimensional space). In this regard, due to re-emission of radiation and capture of scattered radiation, lidar system 230 can be configured to determine a large number of detected data points, which together form a multi-dimensional map (for example, a three-dimensional map) of objects that are near the lidar system 230.

[98] To better illustrate this, further reference is made to FIG. 4 illustrating a bird's eye view representation 400 of a plurality of detected data points 420 in multidimensional space. This multidimensional space may be represented in the coordinate system 1000 of the lidar system 230.

[99] As can be seen in FIG. 4, the lidar system 230 may be configured to generate a plurality of detected data points 420 that together form a multidimensional map of objects near the lidar system 230, i.e., as illustrated, a plurality of detected data points 420 together form a multidimensional map (i) of the outer surface. vehicle 220; and (ii) ground surface 280.

[100] However, it should be noted that if other objects are in the vicinity of the lidar system 230 (in addition to the vehicle 220), then the set of detected data points 420 should form a multidimensional map of (i) the outer surface of the vehicle 220, (iii ) the surface 280 of the earth and (iii) other objects, without departing from the scope of this technology.

[101] With reference to FIG. 10, the coordinate system 1000 of lidar system 230 is described below. Generally speaking, lidar system 230 may use coordinate system 1000 to locate various detected data points in the multidimensional space mentioned above. For example, the coordinate system 1000 of the lidar system 230 may comprise three orthogonal axes, namely the X, Y, and Z axes. In this regard, the coordinate system 1000 of the lidar system 230 may allow the lidar system 230 to transform the set of detected data points 420 in three-dimensional space , having a first dimension along the X axis, a second dimension along the Y axis, and a third dimension along the Z axis.

[102] However, it is contemplated that during acquisition (e.g., when the lidar system 230 generates a plurality of detected data points 420), the lidar system 230 may locate the plurality of detected data points 420 in a spherical coordinate system, and thereafter, the plurality of detected data points The data 420 may be transformed by the lidar system 230 into a coordinate system 1000 having three orthogonal axes as described above.

[103] Returning to the description of FIG. 2, lidar system 230 is operatively coupled to processor 110 of electronic device 210 to transmit data about at least a portion of the vicinity of lidar system 230 (eg, including a plurality of detected data points 420) to processor 110 for processing. Although described in more detail below, data transmitted by lidar system 230 to processor 110 may be used by electronic device 210 to control/maneuver vehicle 220.

[104] It should be noted that a variety of additional sensors and systems may be implemented in combination with lidar system 230 on vehicle 220 in some embodiments of the present technology. For example, camera sensors may be mounted on vehicle 220 and operatively coupled to processor 110. In another example, radar systems may be mounted on vehicle 220 and operatively coupled to processor 110. In this regard, vehicle 220 is illustrated in FIG. 2 for simplicity as having only the lidar system 230; however, in other embodiments, vehicle 220 may be implemented with additional sensors and systems in lidar system 230 without departing from the scope of the present technology.

[105] Lidar system 230 is calibratable.

[106] In the first case, the lidar system 230 may be subject to "internal calibration". Generally speaking, internal calibration refers to technologies that can be used to ensure that lidar system 230 performs properly and data is captured and generated by lidar system 230 accurately. In other words, "internal calibration" may be performed to ensure that data captured or generated by lidar system 230 is not biased as a direct result of malfunction or abnormal operation of lidar system 230.

[107] In one example, lidar system 230 may malfunction or malfunction if it erroneously determines the time-of-flight of radiation, resulting in an erroneous position of an object that scatters the radiation, and thereby results in an erroneously detected data point. Internal calibration is typically performed initially during the manufacture of lidar system 230. Internal calibration of lidar system 230 could also potentially be performed during vehicle 220 upgrades by lidar system 230.

[108] In the second case, the lidar system 230 can also be subjected to "external calibration". Generally speaking, external calibration refers to technologies that can be used to ensure that data provided by lidar system 230 (eg, detected data points forming a multidimensional map of the surroundings of lidar system 230) are properly used by other systems. This means that even if the lidar system 230 is functioning properly (e.g., the lidar system 230 is internally calibrated successfully), the data provided by the lidar system 230 can (albeit without inherent bias) be used by other systems with bias due to a variety of factors. .

[109] Therefore, external calibration is not used to compensate for malfunction or abnormal operation of the lidar system 230 per se, but to compensate for at least part of a variety of factors due to which the data provided by the lidar system 230 can be used through other offset systems.

[110] As should become apparent from the description below, the developers of the present technology create methods and devices for performing external calibration of the lidar system 230. Below in this document, it is described in more detail what factors can be compensated through the methods and devices created by the developers of the present technology, and how they may allow external calibration of the lidar system 230.

Communication network

[111] In some embodiments of the present technology, communication network 245 is the Internet. In alternative non-limiting embodiments, the communications network may be implemented as any suitable local area network (LAN), wide area network (WAN), private communications network, and the like. It should be clearly understood that the communications network implementations are for purposes of illustration only. How the communication link (without a separate reference number) between the electronic device 210 and the communication network 245 is implemented will depend on, among other things, how the electronic device 210 is implemented. Just by way of example, and not as a limitation, in In those embodiments of the present technology in which the electronic device 210 is implemented as a wireless communication device, such as a smartphone or navigation device, the communication link may be implemented as a wireless communication link. Examples of wireless links include, but are not limited to, 3G networks, 4G network links, and the like. Communication network 245 may also use a wireless connection to server 235.

Server

[112] In some embodiments of the present technology, the server 235 is implemented as a server computer and may contain some or all of the components of the computer system 100 of FIG. 1. In one non-limiting example, server 235 is implemented as a Dell™ PowerEdge™ server running the Microsoft™ Windows Server™ operating system, but may also be implemented in any other suitable hardware, software and/or firmware, or a combination of the foregoing. In the illustrated non-limiting embodiments of the present technology, the server is a single server. In alternative non-limiting embodiments of the present technology (not shown), the functionality of the server 235 may be distributed and may be implemented across multiple servers.

[113] In some non-limiting embodiments of the present technology, the processor 110 of the electronic device 210 may communicate with the server 235 to receive one or more updates. Updates may include, but are not limited to, software updates, map updates, route updates, weather updates, and the like. In some embodiments of the present technology, processor 110 may also be configured to transmit certain operational data, such as driving routes, traffic data, performance data, and the like, to server 235. Some or all of the data transmitted between vehicle 220 and server 235 may be encrypted and/or anonymized.

External calibration of the lidar system

[114] As mentioned above, the developers of the present technology create methods and systems for performing external calibration of the lidar system 230 to compensate for at least part of the many factors due to which the data provided by the lidar system 230 may be biased by other systems.

[115] To better illustrate this, assume that the lidar system 230 is mounted on top of the vehicle 220, as seen in FIG. 3.

[116] First, it should be noted that the vehicle 220 has a "forward direction", which is the direction in which the vehicle 220 moves during operation. This means that when the vehicle 220 is moving straight ahead, the vehicle 220 is moving straight along that straight direction. Typically, the forward direction coincides with the longitudinal axis of the vehicle frame 220. In some embodiments, the longitudinal axis of the vehicle frame 220 may also coincide with the axis of symmetry of the vehicle 220 such that portions of the vehicle 220 located on each side of the longitudinal axis of the frame represent are mirror images of each other.

[117] Secondly, it should be noted that the vehicle 220 is moving in this forward direction on the surface 280 of the earth. The ground surface 280, as described above, has a normal direction 285, which is the normal direction to the ground surface 280. For example, at least in some cases, it is assumed that the forward direction of the vehicle 220 may be orthogonal to the direction 285 normal to the surface 280 of the earth.

[118] Third, it should be noted that the lidar system 230 has a coordinate system 1000 that is used to transform the set of detected data points 420 in 3D space relative to the lidar system 230. However, although the set of detected data points 420 are transformed in 3D space relative to the lidar system 230 (in coordinate system 1000), the electronic device 210 does not know a priori, in fact, how the set of detected data points 420 should be transformed in three-dimensional space relative to the vehicle 220.

[119] Therefore, during external calibration of lidar system 230, electronic device 210 may have the task of determining, in a sense, "reference information" about vehicle 220 and/or surroundings 250 in order to transform the set of detected data points 420 about vehicle 220 (and not just with respect to the lidar system 230).

[120] To better illustrate the problem, an example of calibration during vehicle 220 upgrade by lidar system 230 is described below. During vehicle 220 upgrade by lidar system 230, reference information may initially be measured or otherwise determined by a human operator (for example) mounting lidar system 230 on vehicle 220. For example, when lidar system 230 is initially installed on vehicle 220, a human operator can:

- measure or otherwise determine the axis of symmetry of the vehicle 220;

- measure or otherwise determine the direction 285 of the normal to the surface 280 of the earth; And

- install the lidar system 230 on the vehicle 220 in such a way that:

(i) one of the axes of the coordinate system 1000 of the lidar system 230 corresponds to the forward direction of the vehicle 220 (eg, the axis of symmetry of the vehicle 220); And

(ii) another of the axes of the coordinate system 1000 of the lidar system 230 corresponds to the direction 285 of the normal to the surface 280 of the earth.

[121] Such mounting of the lidar system 230 on the vehicle 220 may allow the detected data points of the lidar system 230 to be transformed relative to the vehicle 220 due to the coincidence of (i) some axes of the coordinate system 1000 with (ii) the axis of symmetry of the vehicle 220 and the normal direction 285.

[122] In this regard, in some embodiments, during the upgrade of vehicle 220 by lidar system 230, reference information relative to vehicle 220 (e.g., axis of symmetry of vehicle 220) and relative to surroundings 250 (e.g., direction 285 normal to surface 280 ground) can be measured or otherwise determined by a human operator and used to convert the detected data points of the lidar system 230 relative to the vehicle 220.

[123] However, during operation of vehicle 220, lidar system 230 may inadvertently drift relative to vehicle 220. In one non-limiting example, vehicle 220 may encounter a "bump" in the road it is traveling on. Movement over this "bump" can shift the lidar system 230 relative to the vehicle 220, resulting in an unintended mismatch between (i) the axes of the coordinate system 1000 of the lidar system 230 and (ii) the axis of symmetry of the vehicle 220 and the direction 285 of the normal to the surface 280 of the earth ( for example, reference information). It should be noted that other factors such as weather, for example, may cause this unintended mismatch during operation of the vehicle 220.

[124] In this regard, the developers of the present technology have created systems and methods that, in at least some embodiments of the present technology, are used by the electronic device 210 in order to determine, during the operation of the vehicle 220, reference information regarding the vehicle 220 (e.g., axis of symmetry of vehicle 220) and relative to neighborhoods 250 (e.g., normal direction 285 to ground surface 280) to transform the detected data points of lidar system 230 relative to vehicle 220, and then use the detected data points of lidar system 230 converted with respect to vehicle 220 to control/maneuver vehicle 220.

[125] With reference to FIG. 10, in at least some embodiments of the present technology, the electronic device 210 may be configured, during operation of the vehicle 220:

(i) determine the direction 285 of the normal to the surface 280 of the earth in the coordinate system 1000 of the lidar system 230 (eg, reference information regarding the neighborhood 250)

(ii) determine the first angular displacement 1050 between the normal direction 285 and one of the axes of the coordinate system 1000 (eg, the Z-axis, as illustrated);

(iii) based on the first angular displacement 1050, determine the first rotational shift rule 1060 to tie (eg, align) one of the axes of the coordinate system 1000 with the normal direction 285,

- thereby rotating the set of detected data points 420 in the direction of the first arrow 1070;

(iv) determine the axis of symmetry 800 of the vehicle 220 in the coordinate system 1000 of the lidar system 230 (eg, reference information regarding the vehicle 220);

(v) determine a second angular displacement 1080 between the axis of symmetry 800 of the vehicle 220 and another of the axes of the coordinate system 1000 (eg, the X-axis, as illustrated); And

(vi) based on the second angular displacement 1080, determine a second rotational shift rule 1090 to tie (e.g., align) another of the axes of the coordinate system 1000 with the axis of symmetry 800 of the vehicle 220,

- thereby rotating the set of detected data points 420 in the direction of the second arrow 1100.

[126] Therefore, even if the lidar system 230 is unintentionally misaligned relative to the vehicle 220 during operation, the electronic device 210 may perform the above steps to externally calibrate the lidar system 230, i. in this case, convert the set of detected data points 420 of the lidar system 230 relative to the vehicle 220. In other words, the electronic device 210 may be configured to perform the above steps to compensate for the unintentional displacement of the lidar system 230 relative to the vehicle 220 during operation.

[127] On the other hand, as explained above, external calibration is not used to compensate for malfunction or abnormal operation of the lidar system 230, but instead, to compensate for other factors (such as, for example, unintentionally altered positional relationship between the lidar system 230 and the vehicle 220) such that the detected data points of the lidar system 230 are not used biased by the electronic device 210 to control/maneuver the vehicle 220.

[128] Thus, it can be said that the developers of the present technology create methods and devices for external calibration of the lidar system 230 using (i) the first angular displacement 1050 between one of the axes of the coordinate system 1000 of the lidar system 230 and the normal direction 285 and (ii) a second angular displacement 1080 between another of the axes of the coordinate system 1000 of the lidar system 230 and the axis of symmetry 800 of the vehicle 220.

[129] It is assumed that the first angular displacement 1050 and/or the second angular displacement 1080 can be continuously determined, in real time, during the operation of the vehicle 220 (if possible). In some embodiments, this may allow electronics 210 to continuously compensate, in real time, for unintentional misalignment of lidar system 230 relative to vehicle 220.

[130] As mentioned above, it is contemplated that in some embodiments of the present technology, electronic device 210 may be configured to determine reference information regarding vehicle 220 and/or surroundings 250. In other words, electronic device 210 may be configured to, in during operation of the vehicle 220, determine (i) the axis of symmetry 800 of the vehicle 220 and (ii) the direction 285 of the normal to the surface 280 of the earth. The following describes how the electronic device 210 may be configured to determine a normal direction 285, a first angular displacement 1050, a vehicle axis of symmetry 220, and a second angular displacement 1080 to perform an external calibration of the lidar system 230.

Determination of the direction of the normal to the surface of the earth

[131] In some embodiments of the present technology, the electronic device 210 may be configured to use information regarding at least a portion of the plurality of detected data points 420 illustrated in FIG. 4 to determine the normal direction 285 to the ground surface 280.

[132] In some embodiments, the electronic device 210 may be configured to receive a plurality of detected data points 420 and may be configured to filter out the detected data points that do not correspond to the ground surface 280 and/or determine which detected data points from the sets of detected data points 420 correspond to the surface 280 of the earth.

[133] This can be done by the electronic device 210 in a variety of ways. By way of example only, the detected data points corresponding to the ground surface 280 may share at least one similar characteristic, such as, but not limited to, location in 3D space in a common 2D plane. In this regard, the electronic device 210 may be configured to determine the direction of the normal 285 as the direction of the normal of this common two-dimensional plane in the coordinate system 1000 of the lidar system 230.

[134] However, it is also contemplated that the normal direction 285 can be determined by the electronic device 210 in the coordinate system 1000 in accordance with known techniques in the art.

Determining the axis of symmetry of the vehicle

[135] With reference to FIG. 5 illustrates a plurality of computerized operations 500 that the electronic device 210 may be configured to perform to determine the axis of symmetry 800 of the vehicle 220. The plurality of computerized operations 500 comprise: (i) a filtering operation 502, (ii) a mirroring operation 504, (iii) a 506 projection, (iv) an identification operation 508 of symmetrically opposite detected data points (SODDPI), and (v) an operation 510 of identification of the axes. The plurality of computerized operations 500 are explained in turn below.

[136] As mentioned above, lidar system 230 is configured to generate a plurality of detected data points 420 that together represent a multidimensional map of objects in the vicinity of lidar system 230 (including the outer surface of vehicle 220, ground surface 280, and other potential objects). Electronic device 210 may be configured to receive data indicative of a plurality of detected data points 420 from lidar system 230.

[137] In some embodiments, the electronic device 210 may be configured to perform a filtering operation 502 on a plurality of detected data points 420. Generally speaking, performing filtering operation 502 may allow electronics 210 to determine the subset 600 of detected data points illustrated in FIG. 6, from a plurality of detected data points 420 provided by the lidar system 230 that represents the outer surface of the vehicle 220.

[138] It should be noted that for simplicity, (i) the total number of detected data points in the subset 600 of detected data points that represent the outer surface of the vehicle 220 (as illustrated in FIG. 6) is reduced compared to (ii) the total number of detected data points in the set of detected data points 420 that represent the outer surface of the vehicle 220 (as illustrated in FIG. 4).

[139] In some cases, the filtering operation 502 may be implemented in accordance with known techniques. However, in at least some embodiments, the electronic device 210 may be configured to perform a filtering operation 502 by applying an approximation rule to a given set of detected data points of the lidar system 230. For example, the approximation rule may comprise an indication of a threshold distance, which is considered , includes a subset 600 of detected data points (eg, threshold distance, which is considered to include the outer surface of the vehicle 220). Therefore, detected data points from the set of detected data points 420 of lidar system 230 that are within a threshold distance from lidar system 230 may be identified by electronic device 210 as being part of a subset 600 of detected data points.

[140] Regardless of how the filter operation 502 is implemented, it should be noted that the purpose of the filter operation 502 is to identify which detected data points from the vicinity of the lidar system 230 correspond to the outer surface of the vehicle 220.

[141] However, in other embodiments, the filtering operation 502 may be performed at the same time that the electronic device 210 is configured to filter out detected data points that do not match the ground surface 280 and/or determine which detected data points are from the sets of detected data points 420 correspond to the surface 280 of the earth. In other words, the electronic device 210 may be configured to determine simultaneously from the plurality of detected data points 420, (i) which detected data points correspond to the ground surface 280, and (ii) which detected data points should be included in the subset of detected data points. data representing the outer surface of the vehicle 220.

[142] It is also contemplated that, in at least some embodiments, the filtering operation 502 may be optional and therefore omitted from the set of computer-implemented operations 500 that the electronic device 210 performs to determine the axis of symmetry 800 of the vehicle 220.

[143] In general, it should be noted that the manner in which the electronic device 210 selects/determines a subset of detected data points 600 representing the outer surface of the vehicle 220 from a plurality of detected data points 420 is not particularly limited.

[144] In some embodiments of the present technology, electronic device 210 may also be configured to provide an indication of an increase in the outer surface of vehicle 220. In one non-limiting example, electronic device 210 may be configured to provide such an indication to a human operator during external calibration. lidar system 230. It is contemplated that the electronic device 210 may be configured to provide such an indication for a variety of reasons, such as, for example, in response to a determination by the electronic device 210 that a subset 600 of detected data points does not include a minimum threshold number data points.

[145] Such an indication of an increase in the outer surface of the vehicle 220 may be provided in various forms by the electronic device 210. In one embodiment, providing an indication of an increase in the outer surface of the vehicle 220 may comprise configuring the electronic device 210 to provide indications for a full opening. located opposite the doors of the vehicle 220. In another embodiment, providing an indication of an increase in the outer surface of the vehicle 220 may comprise configuring the electronic device 210 to provide an indication of the full opening of the trunk of the vehicle 220.

[146] Returning to FIG. 5, as mentioned above, the electronic device 210 is also configured to perform a mirroring operation 504. Generally speaking, the electronic device 210 may be configured to perform a mirroring operation 504 to generate a subset 650 of mirror image data points based on a subset 600 of detected data points (see FIG. 6). It should be noted that the electronic device 210, using the mirroring operation 504, may generate a subset 650 of the mirror image data points such that the subset 650 of the mirror image data points represents a mirror image of the outer surface of the vehicle 220.

[147] It is contemplated that by performing the mirroring operation 504, the electronic device 210 can be configured to generate a subset 650 of mirror image data points in a multidimensional space identical to that of the subset 600 of detected data points. In other words, after the mirroring operation 504 is performed by the electronic device 210, the electronic device 210 may have access to both (i) a subset 600 of detected data points representing the outer surface of vehicle 220 and (ii) a subset 650 of mirror data points. image representing a mirror image of the outer surface of the vehicle 220, in the same multi-dimensional space.

[148] To better illustrate this, FIG. 6, illustrated is a virtual mirror axis 680 that electronic device 210 can use to generate a subset 650 of mirror image data points based on a subset 600 of detected data points during mirror operation 504. How the electronic device 210 determines that the mirroring virtual axis 680 is not limiting. It should also be noted that the virtual mirror axis 680 is illustrated by way of illustration only, and as such, it is assumed that the electronic device 210 is not necessarily configured to render a visual image of the virtual mirror axis 680.

[149] As can be seen, the subset 600 of detected data points includes the detected data points 601, 602, 603, 604, 606, and 607. Electronic device 210 may be configured to use mirror virtual axis 680 to generate a subset 650 of mirror image data points such that:

a mirror image data point 657 from a subset of mirror image data points 650 is generated based on the detected data points 607;

a mirror image data point 656 from a subset of mirror image data points 650 is generated based on the detected data points 606;

a mirror image data point 655 from a subset of mirror image data points 650 is generated based on the detected data points 605;

a mirror image data point 654 from a subset of mirror image data points 650 is generated based on the detected data points 604;

a mirror image data point 653 from a subset of mirror image data points 650 is generated based on the detected data points 603;

a mirror image data point 652 from a subset of mirror image data points 650 is generated based on the detected data points 602; And

a mirror image data point 651 from a subset of mirror image data points 650 is generated based on the detected data points 601 .

[150] After the subset 650 of the mirror image data points is generated by the electronic device 210, the electronic device 210 is also configured to perform a projection operation 506 (see FIG. 5). Generally speaking, the electronic device 210 may be configured to perform a projection operation 506 on a subset 650 of mirror image data points such that the subset 650 of mirror image data points "overlaps" the subset 600 of detected data points in a sense. In other words, the electronic device 210 may be configured to perform a projection operation 506 to define data pairs of overlapping data points between (i) a subset 600 of detected data points and (ii) a subset 650 of mirror image data points when a subset of 650 mirror image data points overlaps subset of 600 detected data points.

[151] It is contemplated that during projection operation 506, electronic device 210 may be configured to iteratively minimize the distance between subset 600 of detected data points and subset 650 of mirror image data points (e.g., the distance between subsets 600 and 650 as a whole) to those until the subset 650 of the mirror image data points overlaps the subset 600 of the detected data points. For example, in some embodiments, as part of projection operation 506, electronic device 210 may be configured to apply an iterative nearest point (ICP) algorithm to iteratively minimize the distance between subset 600 of detected data points and subset 650 of mirror image data points.

[152] In some embodiments of the present technology, it is contemplated that iteratively minimizing the distance between subset 600 of detected data points and subset 650 of mirror image data points may comprise executing the electronic device 210 with the ability to define a conversion rule between subset 600 of detected data points and subset 650 of data points mirror image so as to overlap a given mirror image data point from a subset of mirror image data points 650 with a given detected data point from a subset of detected data points 600.

[153] To better illustrate this, with reference to FIG. 7 illustrates output 700 of project operation 506. Note that output 700 is illustrated in FIG. 7 by way of illustration only, and that the electronic device 210 is not necessarily configured to render such a visual image of the output 700.

[154] As can be seen from the output 700, the subset 650 of the mirror image data points overlaps the subset 600 of the detected data points. When the mirror image data point subset 650 overlaps the detected data point subset 600, the electronic device 210 therefore defines data of the overlapping data point pair between (i) the detected data point subset 600 and (ii) the mirror image data point subset 650, such as:

- a first pair 701 of overlapping data points including (i) a detected data point 601 and (ii) a mirror image data point 657;

- a second pair 702 of overlapping data points including (i) a detected data point 603 and (ii) a mirror image data point 656;

- a third pair 703 of overlapping data points including (i) a detected data point 604 and (ii) a mirror image data point 655;

- a fourth pair 704 of overlapping data points including (i) a detected data point 605 and (ii) a mirror image data point 654;

a fifth pair 705 of overlapping data points including (i) a detected data point 606 and (ii) a mirror image data point 653; And

- a sixth pair 706 of overlapping data points including (i) a detected data point 607 and (ii) a mirror image data point 651.

[155] As mentioned above, this output 700 may be generated by electronic device 210 iteratively minimizing the distance between a subset 600 of detected data points and a subset 650 of mirror image data points in an identical multidimensional space. In other words, the electronic device 210 may be configured to perform a sequence of computational iterations in order to generate output 700, wherein, during each iteration of the sequence of computational iterations, the electronic device 210 has the task of minimizing the distance between the subset 600 of detected data points and the subset 650 mirror image data points in identical multidimensional space. On the other hand, as mentioned above, in at least some embodiments of the present technology, generation of data indicative of output 700 may be achieved by electronic device 210 using an ICP algorithm.

[156] It should be noted that, in some cases, not all of the detected data points from the subset 600 of detected data points need to be overlapped by the corresponding mirror image data points from the subset 650 of mirror image data points after the projection operation 506 . For example, as illustrated in FIG. 7, the detected data point 602 of the detected data point subset 600 does not overlap with any mirror image data point of the mirror image data point subset 650.

[157] This also means that, in some cases, not all mirror image data points from the mirror image data point subset 650 need to overlap the corresponding detected data point from the detected data point subset 600 after the projection operation 506 . For example, as illustrated, the mirror image data point 652 from the mirror image data point subset 650 does not overlap the detected data points from the detected data point subset 600.

[158] However, electronic device 210 using projection operation 506 may be configured to iteratively minimize the distance between subset 600 of detected data points and subset 700 of mirror image data points until a threshold number and/or threshold ratio of detected the data points from the subset 600 of the detected data points are not overlaid by the corresponding mirror image data points from the subset 700 of the mirror image data points. It is contemplated that this threshold number and/or this threshold proportion of detected data points may be predetermined by a human operator in at least some embodiments of the present technology.

[159] It should be noted that by performing the projection operation 506 as explained above, the electronic device 210 defines pairs 701, 702, 703, 704, 705, and 706 of overlapping data points. It should also be noted that the data points in any given pair of overlapping data points defined in this manner by the electronic device 210 have a particular relationship to each other.

[160] The relationship between data points in any given pair of overlapping data points is as follows: any given pair of overlapping data points specified by electronic device 210 performing projection operation 506 includes (i) the first data point from a subset 600 of detected data points and (ii) a second data point from a subset of mirror image data points 650, and such that the second data point is a mirror image of a third data point from a subset of detected data points 600 that is symmetrically opposite the first data point in a subset of detected data points 600 .

[161] To better illustrate this relationship, consider the example of the first pair 701 of overlapping data points. As mentioned above, the overlapping data point pair 701 includes (i) a first data point representing a detected data point 601 from a subset 600 of detected data points, and (ii) a second data point representing a mirror image data point 657 from a subset 650 mirror image data points. Recall that the mirror image data point 657 is formed as a mirror image of the detected data point 607 (i.e., the third data point associated with the first overlapping data point pair 701), and since the mirror image data point 657 overlaps the detected data point 601, the detected the data point 607 (ie, the third data point) should be the symmetrically opposite detected data point to the detected data point 601 in the subset 600 of the detected data points.

[162] In other words, the relationship between data points in a given pair of overlapping data points can also be explained as follows: if the detected data point 607 (i.e., the third data point) is not symmetrically opposite the detected data point 601 in the subset 600 of the detected data points , by generating a mirror image data point 657 (i.e., the second data point in the first overlapping data point pair 701) through a mirror operation 504, and then projecting the mirror image data point 657 through a projection operation 506, the mirror image data 657 must not overlap the detected point 601 data (ie, the first data point in the first pair 701 of overlapping data points).

[163] Therefore, it can be said that if a given pair of overlapping data points is specified via the electronic device 210 performing the projection operation 506, then the detected data point (i.e., the first data point) in this given pair of overlapping data points should be located symmetrically opposite another detected data point (i.e., the third data point) in the subset 600 of the detected data points, and that the other detected data point is the data point from which the mirror image data point (i.e., the second data point) is generated in this given pair of overlapping data points.

[164] It should be noted that this relationship between data points in a given pair of overlapping data points is present in each of the pairs 701, 702, 703, 704, 705, and 706 of overlapping data points, as illustrated in the non-limiting example of FIG. 7. This means that:

- since the second pair 702 of overlapping data points is specified by the operation 506 of the projection, in such a case, the detected data point 603 (the first data point in the second pair 702 of overlapping data points) is located symmetrically opposite the detected data point 606 in the subset 600 of detected data points, based on which a mirror image data point 656 (the second data point in the second overlapping data point pair 702) is generated;

- since the third pair 703 of overlapping data points is specified by the projection operation 506, in such a case, the detected data point 604 (the first data point in the third pair 703 of overlapping data points) is located symmetrically opposite the detected data point 605 in the subset 600 of detected data points, based on which a mirror image data point 655 is formed (the second data point in the third overlapping data point pair 703);

- since the fourth pair 704 of overlapping data points is specified by the operation 506 of projection, in such a case, the detected data point 605 (the first data point in the fourth pair of overlapping data points 704) is located symmetrically opposite the detected data point 604 in the subset 600 of detected data points, based on which a mirror image data point 654 (the second data point in the fourth overlapping data point pair 704) is generated;

- since the fifth pair 705 of overlapping data points is specified by the projection operation 506, then the detected data point 606 (the first data point in the fifth pair of overlapping data points 705) is located symmetrically opposite the detected data point 603 in the subset 600 of detected data points, based on which a mirror image data point 653 is formed (the second data point in the fifth overlapping data point pair 705); And

- since the sixth pair of overlapping data points 706 is specified by the projection operation 506, then the detected data point 607 (the first data point in the sixth pair of overlapping data points 706) is located symmetrically opposite the detected data point 601 in the subset 600 of detected data points, based on which a mirror image data point 651 is generated (the second data point in pair 706).

[165] It is contemplated that this relationship between data points in respective pairs 701, 702, 703, 704, 705, and 706 of overlapping data points may be used by electronic device 210 to determine the axis of symmetry of vehicle 220. For example, electronic device 210 may be configured to perform a SODDPI operation 508 (see FIG. 5), during which the electronic device 210 can use this relationship between data points in respective pairs 701, 702, 703, 704, 705, and 706 of overlapping data points.

[166] Generally speaking, during SODDPI operation 508, electronic device 210 is configured to determine which detected data points in detected data point subset 600 are symmetrically opposite each other in detected data point subset 600. As mentioned above, after the pairs 701, 702, 703, 704, 705, and 706 of overlapping data points are specified by project operation 506, electronic device 210 may be configured to determine, during SODDPI operation 508, that:

- detected data points 601 and 607 are symmetrically opposite data points in a subset 600 of detected data points;

- detected data points 603 and 606 are symmetrically opposite data points in a subset 600 of detected data points; And

- detected data points 604 and 605 are symmetrically opposite data points in a subset 600 of detected data points.

[167] It is also true that in some embodiments of the present technology, the electronic device 210 may perform a SODDPI operation 508 to determine which detected data points in the subset 600 of detected data points do not have symmetrically opposite the detected data points in the subset 600 of detected data points. data points.

[168] To better illustrate this, consider an example of a detected data point 602. It should be noted that the detected data point 602 is not part of any pair of overlapping data points that are defined as a result of the projection operation 506 . In other words, the detected data point 602 does not overlap with any mirror image data point from the mirror image data point subset 650 . In this case, the electronic device 210 may be configured to determine during SODDPI operation 508 that the detected data point 602 does not have a detected data point symmetrically opposite the detected data point in the subset 600 of detected data points, because, otherwise, the detected data point 602 data must be overlaid by a mirror image of this symmetrically opposite detected data point after projection operation 506 .

[169] In general terms, during SODDPI operation 508, electronic device 210 may be configured to determine which detected data points in a subset of detected data points have symmetrically opposite detected data points in subset 600 of detected data points (and, optionally, which detected data points do not have symmetrically opposite detected data points).

[170] In some embodiments of the present technology, the electronic device 210 may be configured to use this information as to which detected data points have symmetrically opposite detected data points in a subset 600 of detected data points in order to perform an axes determination operation 510 to determine the axis of symmetry of the vehicle 220. The following describes how the electronic device 210 is configured to perform the axis determination procedure 510 with reference to FIG. 8.

[171] As part of axes determination operation 510, electronics 210 may be configured to select a given pair of symmetrically opposite detected data points and determine a given midpoint of them in multidimensional space. For example, electronic device 210 may be configured to select detected data points 604 and 605 (which are symmetrically opposite each other in a subset 600 of detected data points) and may be configured to determine a first midpoint 802. In another example, electronic device 210 may be configured to select detected data points 603 and 606 (which are symmetrically opposite each other in a subset 600 of detected data points) and may be configured to determine a second midpoint 804. In a further example, electronic device 210 may be configured to select detected data points 601 and 607 (which are symmetrically opposite each other in the subset 600 of detected data points) and may be configured to determine a third midpoint 806.

[172] Also as part of the axis determination procedure 510, the electronic device 210 may also be configured to determine a given line that passes through the midpoints of respective pairs of symmetrically opposite detected data points. In other words, the electronic device 210 may be configured to determine a given line through any combination of at least two of (i) first midpoint 802, (ii) second midpoints 804, and (iii) third midpoint 806. As a result, a given the line may be used by the electronic device 210 as the axis of symmetry 800 of the vehicle 220.

[173] It is contemplated that the electronic device 210 may be configured to determine at least two midpoints between respective pairs of symmetrically opposite detected data points in a subset 600 of detected data points in order to determine a given line passing through at least two midpoints, and use this line as the axis of symmetry of the vehicle 220.

Calibration

[174] As mentioned above, the electronic device 210 may be configured to use the direction 285 of the normal to the surface 280 of the earth and the axis of symmetry 800 of the vehicle 220 to perform an external calibration of the lidar system 230. In other words, the electronic device 210 may be configured to externally calibrate the detected data points of the lidar system 230 by using (i) the normal direction 285 in the coordinate system 1000 of the lidar system 230 and (ii) the axis of symmetry 800 in the coordinate system 1000 of the lidar system 230.

[175] This external calibration is described below, during which the electronic device 210 can be configured to determine externally calibrated detected data points of the lidar system 230.

[176] With reference to FIG. 10, after the electronic device 210 determines the direction 285 of the normal in the coordinate system 1000 of the lidar system 230, the electronic device 210 may determine the first angular displacement 1050 as the angle between the direction of the normal 285 and one of the axes of the coordinate system 1000 (in the illustrated example, the Z axis) .

[177] As mentioned above, the electronic device 210 may be configured to determine the first rotational shift rule 1060 based on the first angular shift 1050 to, in a sense, anchor (e.g., align) one of the axes of the coordinate system 1000 (in the illustrated example, the axes Z) with direction 285 normal. By determining this first rotational shift rule 1060, the electronic device 210 is configured to rotate the plurality of detected data points 420 in the direction of the first arrow 1070.

[178] It should be noted that, after one of the axes of the coordinate system 1000 is aligned with the direction of the normal, the electronic device 210 may be configured to (i) determine the axis of symmetry 800 of the vehicle 220 and (ii) bind (e.g., align) the other of the axes of the coordinate system 1000 (in the illustrated example, the x-axis) with it.

[179] After the electronic device 210 determines the axis of symmetry 800 of the vehicle 220 in the coordinate system 1000 of the lidar system 230, as described above, the electronic device 210 may determine the second angular displacement 1080 as the angle between the axis of symmetry 800 and another of the axes of the system 1000 coordinates (in the illustrated example, the x-axis).

[180] As mentioned above, the electronic device 210 may be configured to determine a second rotational shift rule 1090 based on the second angular shift 1080 to, in a sense, anchor (e.g., align) another of the axes of the coordinate system 1000 (in the illustrated example, the axes X) with an axis of symmetry 800. By defining this second rotational shift rule 1090, the electronic device 210 is configured to rotate the plurality of detected data points 420 in the direction of the second arrow 1100.

[181] In general, it can be said that the electronic device 210 can be configured to determine a first rotational shift rule 1060 (based on the first angular displacement 1050) and a second rotational shift rule 1090 (based on the second angular displacement 1050) for binding one and yet another of the axes of the coordinate system with the normal direction 285 and with the symmetry axis 800, respectively. As a result, the electronic device 210 may be configured to use the first rotational shift rule 1060 and the second rotational shift rule 1090 to rotate the set of detected data points 420, thereby determining an externally calibrated set of detected data points of the lidar system 230.

Computerized methods

[182] In some embodiments, the electronic device 210 may be configured to perform a computerized method 900 to calibrate the lidar system 230. It should be noted that in some embodiments of the present technology, the electronic device 210 may be configured to perform another computerized method to determining the axis of symmetry 800 of the vehicle 220, without departing from the scope of the present technology. It should be noted that another computerized method for determining the axis of symmetry 800 of the vehicle 220 may include at least some of the steps of the computerized method 900, which are explained below in turn.

Step 902 - Obtaining a subset of detected data points

[183] Method 900 begins at 902 by configuring electronic device 210 to obtain a subset 600 of detected data points illustrated in FIG. 6.

[184] In some embodiments, prior to step 902, electronic device 210 may be configured to receive a plurality of detected data points 420 (see FIG. 4) from lidar system 230.

[185] In other embodiments, the electronic device 210 may be configured to filter the plurality of detected data points 420 to determine a subset 600 of the detected data points. In some cases, the electronic device 210 may perform such filtering by applying an approximation rule, as mentioned above, to the set of detected data points 420 of the lidar system 230. For example, the approximation rule may include a threshold distance indication, which is considered to include subset of detected data points.

[186] In other embodiments, prior to step 902, the electronic device 210 may be configured to provide an indication of an increase in the outer surface of the vehicle 220 by (i) providing an indication for the opposite vehicle doors 220 to be fully opened, and/or (ii) providing an indication that the trunk of the vehicle 220 is fully open.

[187] In some embodiments, a subset 600 of detected data points and/or a plurality of detected data points 420 may be obtained in the coordinate system 1000 of the lidar system 230. It is contemplated that the lidar system coordinate system 1000 may have three orthogonal axes, as illustrated in FIG. 10. However, in some cases, it is contemplated that a set of detected data points 420 and/or a subset 600 of detected data points may be collected and/or generated by the lidar system 230 in the spherical coordinate system and thereafter may be converted to the lidar coordinate system 1000 system 230 having three orthogonal axes without departing from the scope of the present technology.

[188] In another further embodiment, the electronic device 210 may also be configured to determine the normal direction 285 to the earth surface 280 based on the plurality of detected data points 420 of the lidar system 230 as mentioned above. The electronic device 210 may also be configured to tie one of the axes of the coordinate system 1000 (in the illustrated example of FIG. 10, the Z-axis) based on the normal direction 285 to the ground surface 280 such that one of the axes extends in the normal direction 285.

Step 904 - Generating a subset of mirror image data points

[189] The method 900 proceeds to step 904 by configuring the electronic device 210 to generate a subset 650 of mirror image data points based on a subset 600 of detected data points (see FIG. 6). As mentioned above, a subset 650 of mirror image data points represents a mirror image of the outer surface of vehicle 220.

[190] It is contemplated that a subset of 650 mirror image data points can be generated by the electronic device 210 in the coordinate system 1000 of the lidar system 230. It is also contemplated that the subset of 650 mirror image data points can be generated in a multidimensional space identical to the multidimensional space of the detected data point subset 600 , for example, in a three-dimensional space identical to the three-dimensional space of a subset 600 of the detected data points.

Step 906 - Projecting a subset of mirror image data points onto a subset of detected data points

[191] The method 900 proceeds to step 906 by configuring the electronic device 210 to project a subset 650 of mirror image data points onto a subset 600 of detected data points and/or vice versa.

[192] With reference to FIG. 7, electronic device 210 may be configured to perform step 906 to define pairs of overlapping data points, such as:

- a first pair 701 of overlapping data points including (i) a detected data point 601 and (ii) a mirror image data point 657;

- a second pair 702 of overlapping data points including (i) a detected data point 603 and (ii) a mirror image data point 656;

- a third pair 703 of overlapping data points including (i) a detected data point 604 and (ii) a mirror image data point 655;

- a fourth pair 704 of overlapping data points including (i) a detected data point 605 and (ii) a mirror image data point 654;

a fifth pair 705 of overlapping data points including (i) a detected data point 606 and (ii) a mirror image data point 653; And

- a sixth pair 706 of overlapping data points including (i) a detected data point 607 and (ii) a mirror image data point 651.

[193] It should be noted that each given pair of overlapping data points includes (i) a first data point from subset 600 of detected data points, and (ii) a second data point from subset 650 of mirror image data points such that the second data point represents is a mirror image of the third data point in the subset 600 of detected data points, which is symmetrically opposite the first data point.

[194] It is contemplated that step 906 may comprise iteratively minimizing, by electronic device 210, the distance between subset 650 of mirror image data points and subset 600 of detected data points. For example, iterative minimization can be performed by electronic device 210 using the ICP algorithm.

[195] It should be noted that, in some embodiments of the present technology, the iterative distance minimization by the electronic device 210 may comprise defining a transformation rule between a subset 650 of mirror image data points and a subset 600 of detected data points such that a given mirror image data point overlaps with a given a detected data point, and wherein this mirror image data point is formed based on this other detected data point, which is located symmetrically opposite this detected data point.

Step 908 - Using the detected data points to determine the axis of symmetry of the vehicle

[196] The method 900 proceeds to step 902 by configuring the electronic device 210 to use (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points to determine the axis of symmetry 800 of the vehicle 220 in the coordinate system 1000 of the lidar system 230.

[197] For example, as illustrated in FIG. 8, electronic device 210 may be configured to use at least two of the following:

(i) detected data points 604 (given first data point) and detected data point 605 (corresponding third data point);

(ii) detected data points 603 (given first data point) and detected data point 606 (corresponding third data point); And

(iii) detected data points 601 (given first data point) and detected data point 607 (corresponding third data point).

[198] It should be noted that, as explained above:

- detected data points 601 and 607 are symmetrically opposite data points in a subset 600 of detected data points;

- detected data points 603 and 606 are symmetrically opposite data points in a subset 600 of detected data points; And

- detected data points 604 and 605 are symmetrically opposite data points in a subset 600 of detected data points.

[199] It should be noted that in at least some embodiments of the present technology, the use of (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite the at least two first data points data, to determine the axis of symmetry 800 of the vehicle 220 may comprise selecting (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite the at least two first data points. Thereafter, for each pair of (i) at least two first data points and (ii) at least two respective third data points, the electronic device 210 may be configured to determine a corresponding midpoint.

[200] For example, electronic device 210 may be configured to select detected data points 604 and 605 (which are symmetrically opposite each other in a subset 600 of detected data points) and may be configured to determine a first midpoint 802. In another example, the electronic device 210 may be configured to select detected data points 603 and 606 (which are symmetrically opposite each other in a subset 600 of detected data points) and may be configured to determine a second midpoint 804. In a further example, electronic device 210 may be configured to the ability to select the detected data points 601 and 607 (which are symmetrically opposite each other in the subset 600 of the detected data points) and can be configured to determine the third midpoint 806.

[201] In some embodiments, the electronic device 210 may also be configured to determine a given line that passes through the midpoints of respective pairs symmetrically opposite the detected data points. In other words, the electronic device 210 may be configured to determine a given line through any combination of at least two of (i) first midpoint 802, (ii) second midpoints 804, and (iii) third midpoint 806. As a result, a given the line may be used by the electronic device 210 as the axis of symmetry 800 of the vehicle 220.

[202] It is contemplated that the electronic device 210 may be configured to determine at least two midpoints between respective pairs of symmetrically opposite detected data points in a subset 600 of detected data points in order to determine a given line passing through at least two midpoints, and use this line as the axis of symmetry of the vehicle 220.

Step 910 - Lidar System Calibration

[203] The method 900 proceeds to step 910 by making the electronic device 210 capable of using for the lidar system 230 a given angular displacement between the axis of symmetry 800 of the vehicle 220 and another of the axes of the coordinate system 1000 of the lidar system 230.

[204] With reference to FIG. 10, in method 900, electronic device 210 may be configured to determine a second angular displacement 1080 between axis of symmetry 800 of vehicle 220 (e.g., a given SDV) and one of the coordinate system 1000 axes of lidar system 230 (in the illustrated example, the X-axis).

[205] In some embodiment, the electronic device 210 may be configured to determine a rotational shift rule 1090 based on the second angular displacement 1080 to rotate the used detected data points of the lidar system 230 (eg, the set of detected data points 420). In this regard, the electronic device 210 is therefore configured to determine the externally calibrated usable detected data points of the lidar system 230.

[206] It is further contemplated that method 900 is further iteratively repeated by electronic device 210 during operation of vehicle 220. In this regard, electronic device 210 may be further configured to iteratively perform external calibration of lidar system 230 during operation of vehicle 220.

[207] Modifications and improvements to the above implementations of the present technology may become apparent to those skilled in the art. The above description is intended to be exemplary and not limiting. As such, the scope of the present technology may be limited solely by the scope of the appended claims.

Claims (116)

1. A method for calibrating a lidar (LIDAR) optical detection and ranging system mounted on an unmanned vehicle (SDV), which is located on the surface of the earth, and the lidar system has a coordinate system, and the coordinate system has axes, and one of the axes runs along the direction of the normal to the surface of the earth, and the method is carried out by means of computer hardware associated with the SDV, the method comprising the steps of: receive by computer hardware from the lidar system a subset of detected data points in the coordinate system of the lidar system and represent the outer surface of the SDV, wherein the subset of detected data points is a subset of the plurality of detected data points of the lidar system; generating, by means of the computer hardware, a subset of the mirror image data points based on the subset of the detected data points, wherein the subset of mirror image data points represents a mirror image of the outer surface of the SDV; projecting by computer hardware a subset of mirror image data points onto a subset of detected data points so as to define pairs of overlapping data points, wherein each given pair of overlapping data points includes: (i) a first data point from a subset of detected data points, and (ii) a second data point from a subset of mirror image data points, such that the second data point is a mirror image of the third data point in the subset of detected data points that is symmetrically opposite the first data point; using computer hardware (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points to determine the axis of symmetry of the SDV in the coordinate system of the lidar system ; And calibrating the lidar system by means of the computer hardware using an angular displacement between the SDV symmetry axis and another of the lidar system coordinate system axes. 2. The method of claim 1, wherein the method further comprises, prior to obtaining a subset of detected data points, the step of: receiving a plurality of detected data points from the lidar system. 3. The method of claim 2, wherein the method further comprises the step of: filtering by the computer hardware the plurality of detected data points to determine a subset of the detected data points. 4. The method of claim 3, wherein filtering the set of detected data points comprises: applying, by means of the computer hardware, an approximation rule to the plurality of detected data points detected by the lidar system. 5. The method of claim 4, wherein the approximation rule comprises: a threshold distance indication, which is considered to include the detected subset of data points. 6. The method of claim 1, wherein the method further comprises, prior to obtaining a plurality of detected data points, the step of: providing by means of the computer hardware an indication to increase the outer surface of the SDV. 7. The method of claim 6, wherein providing an indication of an increase in outer surface comprises the step of: provide by means of the computer hardware an indication for the full opening of the opposite SDV doors. 8. The method of claim 6, wherein providing an indication of an increase in outer surface further comprises: provide by means of the computer hardware an indication to fully open the trunk of the SDV. 9. The method of claim 1, wherein projecting, by the computer hardware, a subset of mirror image data points onto a subset of detected data points, comprising: iteratively minimizing, by means of the computer hardware, a distance between the subset of the mirror image data points and the subset of the detected data points. 10. The method of claim 9, wherein the iterative distance minimization comprises determining a transformation rule between a subset of mirror image data points and a subset of detected data points such that: overlap the given mirror image data point with the given detected data point, wherein the given mirror image data point is formed based on the given other detected point which is located symmetrically opposite the given detected data point. 11. The method of claim 10, wherein the iterative distance minimization comprises: an iterative nearest point (ICP) algorithm is applied by means of the computer hardware. 12. The method of claim 1, wherein (i) at least two first data points and (ii) at least two respective third data points comprise, respectively, two first data points and (ii) two respective third data points. 13. The method of claim 1, wherein (i) at least two first data points and (ii) at least two respective third data points comprise, respectively, three first data points and (ii) three respective third data points. 14. The method of claim. 1, in which the use of (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points, to determine the axis of symmetry SDV, contains steps in which: selecting by computer hardware (i) at least two first data points and (ii) at least two corresponding third data points; for each pair of (i) at least two first data points and (ii) at least two corresponding third data points, a midpoint is determined by the computer hardware; And determining by means of the computer hardware a line through the respective midpoints as the axis of symmetry of the SDV. 15. The method of claim 1, wherein the coordinate system of the lidar system has three orthogonal axes. 16. The method of claim 1, wherein the plurality of detected data points are collected in a spherical coordinate system and converted to a lidar coordinate system having three orthogonal axes. 17. The method of claim 1, wherein the subset of mirror image data points is formed in the coordinate system of the lidar system. 18. The method of claim 10, wherein the subset of mirror image data points is generated in the same multidimensional space as the subset of detected data points. 19. The method of claim 1, wherein the method further comprises, prior to calibration, the steps of: determining, by means of the computer hardware, a normal direction to the earth's surface based on the plurality of detected data points of the lidar system; linking by computer hardware one of the axes of the coordinate system based on the direction of the normal to the earth's surface, so that one of the axes runs in the direction of the normal. 20. The method of claim 1, wherein the method further comprises: determine by means of computer hardware an angular displacement between the axis of symmetry SDV and one of the axes of the coordinate system of the lidar system. 21. The method of claim 20, wherein the use of the angular offset comprises the steps of: determining, by computer hardware, a rotational shift rule based on the angular displacement such that another of the coordinate system axes of the lidar system coincides with the SDV symmetry axis; And using the computer hardware a rotational shift rule to rotate the used detected data points of the lidar system, thereby determining the externally calibrated usable detected data points. 22. The method of claim 1, the method further comprising the step of iteratively repeating the calibration of the lidar system during operation of the SDV. 23. A method for determining the axis of symmetry of an unmanned vehicle (SDV) located on the surface of the earth and containing a lidar (LIDAR) optical detection and ranging system installed on it, moreover, the lidar system has a coordinate system, and the coordinate system has axes, and the method is carried out by means of a computer hardware associated with the SDV, the method comprising the steps of: receive by computer hardware from the lidar system a subset of detected data points in the coordinate system of the lidar system and represent the outer surface of the SDV, wherein the subset of detected data points is a subset of the plurality of detected data points of the lidar system; generating, by means of the computer hardware, a subset of the mirror image data points based on the subset of the detected data points, wherein the subset of mirror image data points represents a mirror image of the outer surface of the SDV; projecting by computer hardware a subset of mirror image data points onto a subset of detected data points so as to define pairs of overlapping data points, wherein each given pair of overlapping data points includes: (i) a first data point from a subset of detected data points, and (ii) a second data point from a subset of mirror image data points, such that the second data point is a mirror image of the third data point in the subset of detected data points that is symmetrically opposite the first data point; using computer hardware (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite at least two first data points to determine the axis of symmetry of the SDV in the coordinate system of the lidar system. 24. Computer hardware for calibrating a lidar (LIDAR) optical detection and ranging system mounted on an unmanned vehicle (SDV), which is located on the surface of the earth, and the lidar system has a coordinate system, and the coordinate system has axes, and one of the axes passes along a direction normal to the earth's surface, wherein the computer hardware is configured to: receive from the lidar system a subset of detected data points in the lidar system coordinate system and represent the outer surface of the SDV, wherein the subset of detected data points is a subset of the plurality of detected data points of the lidar system; generate a subset of mirror image data points based on a subset of detected data points, wherein the subset of mirror image data points represents a mirror image of the outer surface of the SDV; project a subset of mirror image data points onto a subset of detected data points in such a way as to define pairs of overlapping data points, wherein each given pair of overlapping data points includes: (i) a first data point from a subset of detected data points, and (ii) a second data point from a subset of mirror image data points, such that the second data point is a mirror image of the third data point in the subset of detected data points that is symmetrically opposite the first data point; use (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite the at least two first data points to define the axis of symmetry of the SDV in the coordinate system of the lidar system; And calibrate the lidar system using the angular displacement between the SDV symmetry axis and another of the axes of the lidar system coordinate system. 25. The computer hardware of claim 24, wherein the computer hardware is further configured, prior to obtaining a subset of detected data points: receive a plurality of detected data points from the lidar system. 26. The computer hardware of claim 25, wherein the computer hardware is further configured to: filter the set of detected data points to determine a subset of the detected data points. 27. The computer hardware of claim 26, wherein executing the computer hardware with the ability to filter the plurality of detected data points comprises executing the computer hardware with the ability to: apply the approximation rule to the set of detected data points detected by the lidar system. 28. The computer hardware of claim 27, wherein the approximation rule comprises: a threshold distance indication that is considered to include the detected subset of data points. 29. The computer hardware of claim 24, wherein the computer hardware is further configured to, prior to obtaining a plurality of detected data points: provide an indication of an increase in the outer surface of the SDV. 30. The computer hardware of claim 29, wherein in order to provide an indication of an increase in the outer surface, the computer hardware is configured to: provide an indication of the full opening of the opposite SDV doors. 31. The computer hardware of claim 29, wherein in order to provide an indication of an increase in the outer surface, the computer hardware is configured to: provide an indication of the full opening of the SDV trunk. 32. The computer hardware of claim 24, wherein executing the computer hardware with the ability to project a subset of mirror image data points onto a subset of the detected data points comprises executing the computer hardware with the ability to: iteratively minimize the distance between the subset of mirror image data points and the subset of detected data points. 33. The computer hardware of claim 32, wherein in order to iteratively minimize the distance, the computer hardware is configured to determine a transformation rule between the subset of mirror image data points and the subset of detected data points such that: overlap the given mirror image data point with the given detected data point, wherein the given mirror image data point is formed based on the given other detected point which is located symmetrically opposite the given detected data point. 34. The computer hardware of claim 33, wherein in order to iteratively minimize the distance, the computer hardware is configured to: apply the iterative nearest point (ICP) algorithm. 35. The computer hardware of claim 24, wherein (i) at least two first data points and (ii) at least two respective third data points comprise, respectively, two first data points and (ii) two respective third data points data. 36. The computer hardware of claim 24, wherein (i) at least two first data points and (ii) at least two respective third data points comprise, respectively, three first data points and (ii) three respective third data points data. 37. The computer hardware of claim 24, wherein using (i) at least two first data points and (ii) at least two corresponding third data points that are symmetrically opposite the at least two first data points, to determine axis of symmetry SDV contains a computer hardware implementation with the ability to: select (i) at least two first data points and (ii) at least two corresponding third data points; for each pair of (i) at least two first data points and (ii) at least two corresponding third data points, determine a midpoint; And define the line through the respective midpoints as the axis of symmetry of the SDV. 38. The computer hardware of claim 24, wherein the coordinate system of the lidar system has three orthogonal axes. 39. The computer hardware of claim 24, wherein the plurality of detected data points are collected in a spherical coordinate system and converted to a lidar coordinate system having three orthogonal axes. 40. The computer hardware of claim 24, wherein the subset of mirror image data points is generated in the coordinate system of the lidar system. 41. The computer hardware of claim 33, wherein the subset of mirror image data points is generated in the same multidimensional space as the subset of detected data points. 42. The computer hardware of claim 24, wherein the computer hardware is further configured to, prior to calibration: determine the direction of a normal to the earth's surface based on the plurality of detected data points of the lidar system; snap one of the axes of the coordinate system based on the direction of the normal to the earth's surface, so that one of the axes runs in the direction of the normal. 43. The computer hardware of claim 24, wherein the computer hardware is further configured to: determine the angular displacement between the SDV symmetry axis and one of the coordinate system axes of the lidar system. 44. The computer hardware of claim 43, wherein the use of the angular displacement comprises executing the computer hardware with the ability to: determine a rotational shift rule based on the angular shift such that another of the coordinate system axes of the lidar system coincides with the SDV symmetry axis; And use the rotational shift rule to rotate the used detected data points of the lidar system, thereby determining the externally calibrated usable detected data points. 45. The computer hardware of claim 24, wherein the computer hardware is further configured to iteratively calibrate the lidar system during operation of the SDV.

Images