JP2021108137A - Marking method and device of navigation target, electronic facilities, and computer-readable medium - Google Patents

Abstract

To provide a method and a device for marking a navigation target for reducing a mark area of a mark image, reducing a mark range, and increasing mark efficiency, and to provide electronic facilities, a computer-readable medium, and a computer program product.SOLUTION: A method includes steps of: acquiring a mark image; reducing a mark area on the mark image on the basis of a mark object; and creating a mark processing model for matching to the mark object on the basis of the mark object and the corresponding mark area. The mark processing model reduces the mark area on the mark image.SELECTED DRAWING: Figure 1

Description

The examples in this publication relate to the technical field of augmented reality navigation, in particular to navigation target marking methods and devices, electronics, computer readable storage media and computer program products.

Augmented Reality (abbreviated as AR) is a new technology developed based on virtual reality, a technology that enhances the user's perception of the real world by the information provided by the computer system, and realizes virtual information. It is applied to the world, and augmented reality is realized by superimposing the presentation information of virtual objects, scenes or systems generated by computers on the real scene.

In navigation that expands reality, it is necessary to detect and mark AR navigation data, for example, to mark information such as vehicles in front, pedestrians, and road signs. Manual marking is a normal marking method, and this marking method has low processing efficiency and requires a large amount of labor costs. In addition, as the amount of data increases, the probability of making a mistake increases, which greatly affects the health condition such as eyesight.

The embodiments of this publication provide navigation target marking methods and devices, electronic equipment, computer readable storage media and computer program products.

In the first aspect, the embodiments of this publication are
Steps to get the mark image and
A step of reducing the mark area in the mark image based on the mark target, and
Including a step of creating a mark processing model matching the mark target based on the mark target and the corresponding mark area.
The mark processing model provides a method of marking a navigation target, which reduces the mark area in the mark image.

In some embodiments, the step of reducing the mark area in the mark image based on the mark target is
The step of determining the vanishing point in the mark image and
A step of determining the lowest mark point and the highest mark point based on the vanishing point and the mark target, and
The step includes determining the sealed area surrounded by the lowest mark point and the highest mark point as the mark area.

In some embodiments, the step of determining the lowest and highest mark points based on the vanishing points and marking targets is
When the mark target is a vehicle, the lowest mark point includes a first intersection where the base of the mark image intersects the left side, and a second intersection whose bottom intersects the right side, and the highest mark point. Refers to a third intersection where the continuous line between the vanishing point and the first intersection intersects the far end line, and a fourth intersection where the continuous line between the vanishing point and the second intersection intersects the far end line. Including, the far end line includes a line that is parallel to the base of the mark image and has a preset distance from the base of the mark image.

In some embodiments, the step of determining the lowest and highest mark points based on the vanishing points and marking targets is
When the mark target is a pedestrian, the lowest mark point includes a first intersection where the base of the mark image intersects the left side and a second intersection whose bottom intersects the right side, and the highest mark. The points include a third intersection where the altitude line intersects the left side of the mark image and a fourth intersection where the altitude line intersects the right side of the mark image, and the altitude line has the vanishing point. Includes that the line passes through and the distance to the ground is a preset height.

In some embodiments, the step of determining the lowest and highest mark points based on the vanishing points and marking targets is
When the mark target is a traffic sign, the lowest mark points are the first intersection where the first altitude line intersects the left side of the mark image, and the first altitude line is the right side of the mark image. The highest mark point includes a second intersection that intersects, a third intersection where the second altitude line intersects the left side of the mark image, and a fourth intersection where the second altitude line intersects the right side of the mark image. The first altitude line is a line having a first preset height that passes through the vanishing point and is at a distance from the ground, and the second altitude line passes through the vanishing point and is with the ground. The distance is a second preset height line, including that the first preset height is smaller than the second preset height.

In some embodiments, after the step of creating a mark processing model that matches the mark target based on the mark target and the corresponding mark region.
The step of acquiring the thermodynamic chart of the mark region based on the mark processing model, and
Includes a step of marking a target in the marked area based on the thermodynamic chart.

In some embodiments, prior to the step of reducing the mark area in the mark image based on the mark target,
Further including a step of excluding the approximate image in the mark image.

In some embodiments, prior to the step of reducing the mark area in the mark image based on the mark target,
The step of acquiring the latitude and longitude information of the location of the mark image and
It includes a step of determining the mark target based on the latitude and longitude information.

In the second aspect, the embodiments of this publication are
The acquisition module for acquiring the mark image and
A mark area reduction module for reducing the mark area in the mark image based on the mark target,
Includes a mark processing model acquisition module that creates a mark processing model that matches the mark target based on the mark target and the corresponding mark area.
The mark processing model provides a mark device for a navigation target for reducing a mark area in the mark image.

In some embodiments, the mark area reduction module
A vanishing point determining means for determining the vanishing point in the mark image, and
A mark point determination means for determining the lowest mark point and the highest mark point based on the vanishing point and the mark target, and
A mark area reducing means for determining a closed area surrounded by the lowest mark point and the highest mark point as the mark area is included.

In some embodiments, the device is
An approximate image exclusion module for excluding the approximate image in the mark image is further included.

In some embodiments, the device is
A latitude and longitude determination module for acquiring latitude and longitude information of the location of the mark image,
It further includes a mark target determination module for determining the mark target based on the latitude and longitude information.

In the third aspect, the examples of this publication are
With one or more processors
A method of marking the arbitrary navigation target on the one or more processors by storing one or more programs and executing the one or more programs on the one or more processors. With the memory that realizes
Provided is an electronic facility including one or more I / O interfaces connected between the processor and the memory and arranged to realize information alternating between the processor and the memory.

In the fourth aspect, the embodiments of this publication are
A computer program is stored, and when executed by the processor, the computer program provides a computer-readable storage medium that realizes the method of marking an arbitrary navigation target.

In the fifth aspect, the embodiments of this publication are
Includes a computer program that implements the method of marking any of the navigation targets described above when executed by a processor.
Providing computer program products.

The navigation target marking method provided by the present embodiment includes a step of acquiring a mark image, a step of reducing a mark area in the mark image based on the mark target, and the mark target and the corresponding mark. The mark processing model includes a step of creating a mark processing model that matches the mark target based on the area, and the mark processing model is for reducing the mark area in the mark image. The mark method can reduce the mark area in the mark image, shorten the mark range, improve the mark efficiency, reduce the labor intensity, reduce the influence on the health of the mark personnel, and reduce the error rate of the mark. can.

The drawings are provided to better understand the embodiments of the present publication, form part of the specification, and are used in the description of the present publication together with the examples of the present publication and restrict the present publication. is not it. By explaining a more detailed exemplary embodiment with reference to the drawings, the above contents and other features and merits will become more obvious to those skilled in the art.

FIG. 1 is a flowchart of a navigation target marking method provided by the first embodiment of the present publication. FIG. 2 is a flowchart of step 102 in the navigation target marking method provided by the first embodiment of the present publication. FIG. 3 is a mark conceptual diagram in which the mark target is a vehicle in the embodiment of this publication. FIG. 4 is a conceptual diagram of a mark whose mark target is a vehicle in the embodiment of this publication. FIG. 5 is a mark conceptual diagram in which the mark target in the embodiment of this publication is a traffic sign. FIG. 6 is a flowchart of a navigation target marking method provided by the second embodiment of the present publication. FIG. 7 is a flowchart of a navigation target marking method provided by the third embodiment of the present publication. FIG. 8 is a principle block diagram of the navigation target marking device provided by the fourth embodiment of the present publication. FIG. 9 is a principle block diagram of the mark area reduction module in the mark device of the navigation target provided by the fourth embodiment of the present publication. FIG. 10 is a principle block diagram of the navigation target marking device provided by the fifth embodiment of the present publication. FIG. 11 is a principle block diagram of the navigation target marking device provided by the sixth embodiment of the present publication. FIG. 12 is a block diagram of the electronic equipment provided by the examples of the present disclosure.

Specific embodiment

To help those skilled in the art better understand the technical proposals of this publication, the drawings are combined below to provide a detailed description of the navigation target marking methods and devices, electronics, computer readable storage media and computer program products provided by this publication. do.

In the latter part, an exemplary embodiment will be described more fully with reference to the drawings. The exemplary examples can be embodied in different forms, but should not be construed as limited to the examples described in the text. Rather, the purpose of providing these examples is to make this publication clear and complete so that those skilled in the art can fully understand the scope of this publication.

As long as there is no contradiction, each embodiment of this publication and each feature in the embodiment can be combined with each other.

For example, the term "and / or" used in the text includes any and all combinations of one or more of the related items listed.

The terminology used in this text is merely to describe a particular embodiment and does not limit this publication. For example, the singular forms "one" and "corresponding" used in the text are intended to include multiple forms, unless otherwise explicitly stated in the preceding and following sentences. It should be noted that, in the present specification, when the description "including" and / or "consisting of ..." is used, it refers to the existence of the above-mentioned features, whole, steps, operations, members and / or components, but one or more. Does not preclude the existence or addition of multiple other features, whole, steps, operations, parts, components and / or groups thereof.

Unless otherwise specified, the meanings of all terms (technical and scientific terms) used in the text are the same as those commonly understood by those skilled in the art. Unless expressly limited by the present invention, for example, terms limited in a common dictionary should be construed as meanings that are consistent with the meanings in the context of the relevant technology and the present publication, and are idealized or overly formalized. It is not interpreted as a general meaning.

The navigation target marking methods and devices provided in this published example pre-process navigation data to assist manual marking, improve marking efficiency, and reduce labor costs and health impacts. Can be done.

In a first aspect, the published embodiments provide a method of marking navigation targets. The mark method can assist the manual mark, and by reducing the range of the manual mark, the labor intensity of the manual mark can be reduced, the mark efficiency can be improved, and the error rate of the mark can be reduced.

FIG. 1 is a flowchart of a navigation target marking method provided by the first embodiment of the present publication. With reference to Figure 1, the method of marking a navigation target involves the following steps:

Step 101: Get the mark image.

In the process of AR navigation, at least one camera placed in the vehicle can be used to collect a real-time image of the current driving environment, that is, an image of the surroundings of the vehicle in real time. Each actual scene image can be a mark image. There may be multiple mark targets in the vehicle surrounding image collected at this stage. These mark targets belong to different types, such as vehicles, pedestrians, traffic signs, roads, buildings, and the like. Since mark targets of different types have different characteristics, it is necessary to determine different mark areas for different mark types and use different methods before marking the mark image.

Step 102: Reduce the mark area in the mark image based on the mark target.

In the examples of the present publication, since the appearance rate of different mark targets in different regions in the mark image is different, the region of the mark image that people pay attention to is also different. Therefore, in the embodiment of the present disclosure, the mark area in the mark image is reduced based on the mark target.

For example, when the mark target is a vehicle, the appearance rate of the vehicle in the central region of the mark image is higher than that in the marginal region. When the mark target is a traffic sign, the appearance rate of the traffic sign in the marginal region of the mark image is larger than that in the central region.

Step 103: Create a mark processing model that matches the mark target based on the mark target and the corresponding mark area.

In step 103, there is a correspondence between the mark target and the mark area because the appearance rate of the mark target in the mark image and the degree of attention of the people to the different areas of the mark image are different. Therefore, a mark processing model that matches the mark target is created based on the mark target and the corresponding mark area. The mark processing model will be introduced later.

In the embodiment of this publication, a mark processing model for different mark targets can be created based on the mark target and the mark area, the mark area in the mark image can be reduced, and the amount of manual mark work can be further reduced later. However, labor intensity can be reduced and mark efficiency can be improved.

FIG. 2 is a flowchart of step 102 in the navigation target marking method provided by the first embodiment of the present publication. As shown in FIG. 2, the step of reducing the mark area in the mark image based on the mark target includes the following steps.

Step 201: Determine the vanishing point in the mark image.

In reality, it can be observed with the naked eye that parallel lines can be focused on the vanishing point, just as the two rails of a railroad appear to converge on the horizon. Therefore, the vanishing point in the examples of this publication is the point until two or more parallel lines extend to a distant horizon (HORIZON LINE) and are focused.

In step 201, the vanishing point in the mark image is determined, and the vanishing point in the mark image can be detected and determined by the Hough transform or the RANSAC method. For example, two straight lines are randomly selected to create a virtual vanishing point, and then the number of straight lines passing through this virtual vanishing point is calculated. After a certain number of iterations, the return value is the vanishing point that maximizes the number of intersecting straight lines.

Step 202: Determine the lowest and highest mark points based on the vanishing points and marking targets.

Since the marks are different, the positions of the lowest mark point and the highest mark point are different. Therefore, in step 202, the mark area is determined by the lowest mark point and the highest mark point.

In some embodiments, the angle at which the mark image is taken is different, and the vanishing point position in the mark image is different. Therefore, in step 202, the lowest mark point and the highest mark point are determined depending on the vanishing point and the mark target, and then the minimum The mark area is determined by the mark point and the highest mark point.

In step 203, the closed area surrounded by the lowest mark point and the highest mark point is determined as the mark area.

In the following, the method of determining the lowest mark point and the highest mark point in the mark image will be introduced in detail for different mark targets.

FIG. 3 is a mark conceptual diagram in which the mark target is a vehicle in the embodiment of this publication. In FIG. 3, the angle of the observer or the camera is used as the coordinates, the left side of the observer is the left side of the mark image, the right side of the observer is the right side of the mark image, and the upper side of the observer's line of sight (the top of the paper) is the front side of the mark image. , The lower part of the observer's line of sight (the bottom of the paper) is the lower side of the mark image.

As shown in FIG. 3, when the mark target is a vehicle, the lowest mark points are the first intersection A where the bottom side (lower side) of the mark image intersects the left side, and the bottom side intersects the right side. Includes second intersection B. The highest mark points are the third intersection G where the continuous line OA of the vanishing point O and the first intersection A intersects the far end line GH, and the continuous line OB of the vanishing point O and the second intersection B intersects the far end line GH. Includes the 4th intersection H.

Among them, the far end line GH is a line that is parallel to the base of the mark image and the distance from the base of the mark image is a preset distance S. In some embodiments, the preset distance S can be set according to the actual situation. For example, the preset distance S is 20 meters, 30 meters or 50 meters. The actual size of the preset mark image of the distance S can be determined by the conventional method, and is not limited in the examples of the present disclosure.

When the mark target is a vehicle, it is not difficult to understand that the lowest mark point includes the first intersection A and the second intersection B. The highest mark points include the third intersection G and the fourth intersection H. Therefore, the mark area is a trapezoidal ABGH area surrounded by the first intersection A, the second intersection B, the third intersection G, and the fourth intersection H. When further marking the mark image, mosaic processing is performed on the areas other than the mark area ABGH to mark only the mark area ABGH, and the mark area is significantly reduced.

FIG. 4 is a conceptual diagram of a mark whose mark target is a vehicle in the embodiment of this publication. In FIG. 4, the left side, the right side, the front side and the lower side of the mark image are the same as the definitions in FIG. 3, and are not described again here.

As shown in FIG. 4, when the mark target is a pedestrian, the lowest mark points are the first intersection A where the base of the mark image intersects the left side and the second intersection B where the bottom intersects the right side. including. The highest mark points include a third intersection C where the altitude line OC intersects the left side of the mark image and a fourth intersection D where the altitude line OD intersects the right side of the mark image.

Among them, the altitude lines OC and OD are lines that pass through the vanishing point O and have a preset height with respect to the ground. In some embodiments, the preset height can be set according to the actual situation. For example, the preset height is 1.8 meters, 2 meters or 2.3 meters. In the mark image, the pedestrians of interest are almost below the altitude line OC and OD. The actual size of the preset height in the mark image can be determined by a conventional method, and is not limited in the examples of the present disclosure.

When the mark target is a pedestrian, it is not difficult to understand that the lowest mark point includes the first intersection A and the second intersection B. The highest mark points include the third intersection C and the fourth intersection D. Therefore, the mark area is an ABCD area surrounded by the first intersection A, the second intersection B, the third intersection C, and the fourth intersection D. When further marking the mark image, mosaic processing is performed on the area other than the mark area ABCD to mark only the mark area ABCD, and the mark area is significantly reduced.

FIG. 5 is a conceptual diagram of a mark whose mark target is a traffic sign in the embodiment of this publication. In FIG. 5, the left side, the right side, the front side and the lower side of the mark image are the same as the definitions in FIG. 3, and are not described again here.

As shown in FIG. 5, when the mark target is a traffic sign, the lowest mark points are the first intersection E where the first altitude line OE intersects the left side of the mark image, and the first altitude line OF is the mark image. The highest mark points include the second intersection F that intersects the right side of the mark image, the third intersection G where the second altitude line OG intersects the edge of the mark image, and the second altitude line OH is the side of the mark image. Includes the 4th intersection H that intersects the edge.

Among them, the first altitude lines OE and OF pass through the vanishing point O, and the lines whose distance to the ground is the first preset height H1 and the second altitude lines OG and OH pass through the vanishing point O. , A line whose distance to the ground is the second preset height H2. The first preset height H1 is smaller than the second preset height H2. In some embodiments, the first preset height H1 and the second preset height H2 can be set according to the actual situation. For example, the first preset height H1 is 1.5 meters and the second preset height H2 is 2 meters. In the mark image, the traffic sign of interest is between approximately 1.5 and 2 meters. The actual dimensions of the first preset height H1 and the second preset height H2 in the mark image can be determined by a conventional method, and are not limited in the examples of this publication.

When the mark target is a traffic sign, it is not difficult to understand that the lowest mark point includes the first intersection E and the second intersection F. The highest mark points include the third intersection G and the fourth intersection H. Therefore, the mark area is an EFGH area surrounded by the first intersection E, the second intersection F, the third intersection G, and the fourth intersection H. When further marking the mark image, mosaic processing is performed on the areas other than the mark area EFGH to mark only the mark area EFGH, and the mark area is significantly reduced.

The point to be explained is that the third intersection G may be on the left side of the mark image, it may be on the top (top side) of the mark image, and the fourth intersection H may be on the left side of the mark image. It may be on the right side, or it may be on the top of the mark image. When the 3rd intersection G and the 4th intersection H are at the top of the mark image, the EFGH area surrounded by the 1st intersection E, the 2nd intersection F, the 3rd intersection G and the 4th intersection H is extended to the CDEF area. The CDEF area may be a mark area.

The point to be explained is that all of the above three mark area selections can collectively cover the data sets of the same collection environment, that is, all the mark images collected by the same camera are the above mark areas. Adapts to the selection of, which reduces the mark area.

In some embodiments, a mark processing model is created based on the mark target and the corresponding mark area, a thermodynamic chart of the mark area is obtained based on the mark processing model, and finally, a thermodynamic chart. Marks the target of the mark area in the mark image based on.

When the mark target is a vehicle, an attention model is created based on the mark target and the corresponding mark area. Create an attention thermodynamic chart using the attention model and mark the target in the marked area based on the thermodynamic chart.

For example, an attention thermodynamic chart is created based on an attention model, and the attention thermodynamic chart is advantageous for marking a mark image. In the attention thermodynamic chart, the closer to the red area, the more likely the vehicle will appear, which can reduce the labor intensity of the mark.

When the mark target is a pedestrian, a pedestrian mark model that matches the pedestrian is created based on the pedestrian and the pedestrian mark area.

Pedestrians are pre-marked by combining a Support Vector Machine (abbreviated as SVM) model with a Histogram of Oriented Gradient (abbreviated as HOG). Specifically, 1000 positive samples and 1000 negative samples are used as training samples, feature extraction is performed, the HOG features of the samples are calculated, and finally model training is performed using the HOG features as SVM inputs, and pedestrians. Get the mark model.

When the mark target is a traffic sign, a traffic sign model is generated based on the color model, that is, the type of the traffic sign is identified based on the color and shape. For example, a red circle is a prohibited traffic sign, a yellow triangle is a warning traffic sign, and a blue square is an instruction traffic sign. Specifically, red, yellow, and blue areas are selected by a color model (abbreviated as Hue, Saturation, Value, HSV), and mosaic processing is performed on areas other than these three colors (black or white). ), Then calculate the shape of each area and mark the traffic signs based on the shape and color. For example, if the shape is circular and the color is red, the area is roughly marked as a prohibited traffic sign. If the shape is triangular and the color is yellow, the area is roughly marked as a warning traffic sign. If the shape is rectangular and the color is blue, the area is roughly marked as a traffic sign for instructions.

FIG. 6 is a flowchart of a navigation target marking method provided by the second embodiment of the present publication. With reference to FIG. 6, the method of marking the navigation target includes the following steps.

Step 601: Get the mark image.

Of these, the step of acquiring the mark image is the same as step 101 in the first embodiment, and therefore will not be described again here.

Step 602: Eliminate the approximate image in the mark image.

In practical applications, there are many images with similar similarities in the mark images acquired by the camera. By eliminating mark images whose similarity falls within the similarity threshold before the mark, the number of mark images is reduced and the efficiency of the mark is improved.

In step 602, the Hamming distance between all the mark images is calculated using the difference value hash algorithm, the similarity threshold is set as the Hamming distance, and the image in the section where the Hamming distance is [S1, S2] is set as the approximate image. do.

For example, the similarity threshold is set to the Hamming distances S1 = 0 and S2 = 10, the Hamming distances of all the mark images are calculated, and the mark images whose Hamming distances are within [0, 10] are regarded as approximate images. The approximate image is excluded, and only one of the approximate images is used as the mark image.

Step 603: Reduce the mark area in the mark image based on the mark target.

Among them, the step of reducing the mark area in the mark image based on the mark target is the same as step 102 in the first embodiment, and will not be described again here.

Step 604: Create a mark processing model that matches the mark target based on the mark target and the corresponding mark area. Among them, the mark processing model is for reducing the mark area in the mark image.

Of these, the mark processing model creation step is the same as step 103 in the first embodiment, and will not be described again here.

FIG. 7 is a flowchart of a navigation target marking method provided by the third embodiment of the present publication. With reference to Figure 7, the method of marking a navigation target involves the following steps:

Step 701: Get the mark image.

In step 701, the step of acquiring the mark image is the same as that of step 101 in the first embodiment, and will not be described again here.

Step 702: Eliminate the approximate image in the mark image.

In step 702, the step of eliminating the approximate image is the same as step 602 in the second embodiment, and will not be described again here.

Step 703: Acquire the latitude and longitude information of the location of the mark image.

In step 703, the vehicle is equipped with a camera and a positioning system, the camera is for acquiring the mark image, and the positioning system can acquire the latitude and longitude information of the location position of the mark image. Among them, the positioning system can adopt a general-purpose positioning system, and the present invention is not limited thereto.

Step 704: Determine the mark target based on the latitude and longitude information.

In step 704, the scene of the mark image can be determined based on the latitude and longitude information, and by determining the mark target based on the scene, the mark target is reduced and the mark efficiency is improved.

For example, when determining whether the scene of the mark image is a suburb or a distant region based on the latitude and longitude information, the mark target may be a traffic sign and a vehicle. When determining that the scene of the mark image is in the city based on the latitude and longitude information, the mark target may be a pedestrian, a vehicle, and a traffic sign.

Step 705: Reduce the mark area in the mark image based on the mark target.

In step 705, the step of reducing the mark area in the mark image based on the mark target is the same as step 102 in the first embodiment, and will not be described again here.

Step 706: Create a mark processing model that matches the mark target based on the mark target and the corresponding mark area. Among them, the mark processing model is for reducing the mark area in the mark image.

In step 706, the step of creating the mark processing model is the same as step 103 in the first embodiment, and will not be described again here.

The point to be explained is that, in the embodiment of the present invention, the step of excluding the approximate image in the mark image is set in step 402, and the step of acquiring the latitude and longitude information of the location position of the mark image is set in step 403. However, the examples of the present invention are not limited to this, and swapping the order of steps 402 and 403 also belongs to the claims of the present invention.

The navigation target marking method provided by the present embodiment is based on a step of acquiring a mark image, a step of reducing the mark area in the mark image based on the mark target, and a mark target and the corresponding mark area. The mark processing model includes a step of creating a mark processing model that matches the mark target, and the mark processing model is for reducing the mark area in the mark image. The mark method can reduce the mark area in the mark image, shorten the mark range, improve the mark efficiency, reduce the labor intensity, reduce the influence on the health of the mark personnel, and reduce the mark error rate. ..

In a second aspect, a fourth embodiment of the present disclosure provides a navigation target marking device. The mark method can assist the manual mark, and by reducing the range of the manual mark, the labor intensity of the manual mark can be reduced, the mark efficiency can be improved, and the error rate of the mark can be reduced.

FIG. 8 is a principle block diagram of the navigation target marking device provided by the fourth embodiment of the present publication. Referring to FIG. 8, the navigation target marking device includes the following modules.

Acquisition module 801: This is for acquiring a mark image.

The method by which the acquisition module 801 acquires the mark image is the same as that in step 101 in the first embodiment, and will not be described again here.

Mark area reduction module 802: This is for reducing the mark area in the mark image based on the mark target.

Since the appearance rates of different mark targets in different areas in the mark image are different, the areas of the mark images that people pay attention to are not the same, so the mark area reduction module 802 reduces the mark area in the mark image based on the mark target. do.

For example, when the mark target is a vehicle, the appearance rate of the vehicle in the central region of the mark image is higher than that in the marginal region. When the mark target is a traffic sign, the appearance rate of the traffic sign in the marginal region of the mark image is larger than that in the central region.

Mark processing model acquisition module 803: This is for creating a mark processing model that matches the mark target based on the mark target and the corresponding mark area. Among them, the mark processing model is for reducing the mark area in the mark image.

FIG. 9 is a principle block diagram of the mark area reduction module in the mark device of the navigation target provided by the fourth embodiment of the present publication. As shown in FIG. 9, the mark area reduction module includes the following means.

Vanishing point determining means 901: This is for determining the vanishing point in the mark image.

The vanishing point in the mark image can be detected and confirmed by the Hough transform or RANSAC method. For example, two straight lines are randomly selected to create a virtual vanishing point, and then the number of straight lines passing through this virtual vanishing point is calculated. After a certain number of iterations, the return value is the vanishing point that maximizes the number of intersecting straight lines.

Mark point determination means 902: This is for determining the lowest mark point and the highest mark point based on the vanishing point and the mark target.

Among them, the mark point determining means 902 includes the lowest mark point sub-means and the highest mark point sub-means, and the marking methods of the lowest mark point sub-means and the highest mark point sub-means relate to the mark target.

In some embodiments, the lowest mark point sub-means is a first intersection A where the base of the mark image intersects the left side and a second intersection where the bottom intersects the right side when the mark target is a vehicle. This is for marking B as the lowest mark point.

The highest mark point sub-means is the third intersection G where the continuous line OA of the vanishing point O and the first intersection A intersects the far end line GH when the mark target is a vehicle, and the vanishing point O and the second intersection B. This is for marking the fourth intersection H where the continuous line OB intersects the far end line GH as the highest mark point.

Among them, the far end line GH is a line that is parallel to the base of the mark image and the distance from the base of the mark image is a preset distance S.

In some embodiments, the lowest mark point sub-means is a first intersection A where the base of the mark image intersects the left side and a second where the bottom intersects the right side when the mark target is a pedestrian. This is for marking the intersection B as the lowest mark point.

The highest mark point sub-means is the third intersection C where the altitude line OC intersects the left side of the mark image when the mark target is a pedestrian, and the fourth intersection where the altitude line OD intersects the right side of the mark image. This is for marking the intersection D as the highest mark point.

Of these, the altitude lines OC and OD are lines that pass through the vanishing point and have a preset height of distance from the ground.

In some embodiments, the lowest mark point sub-means is the first intersection E, where the first altitude line OE intersects the left side of the mark image, and the first altitude line OF, when the mark target is a traffic sign. This is for marking the second intersection F, which intersects the right side of the mark image, as the lowest mark point.

The highest mark point sub-means is the third intersection G where the second altitude line OG intersects the edge of the mark image, and the fourth intersection H where the second altitude line OH intersects the edge of the mark image as the highest mark point. It is for marking.

Among them, the first altitude lines OE and OF pass through the vanishing point O, and the lines whose distance to the ground is the first preset height H1 and the second altitude lines OG and OH pass through the vanishing point O. , A line whose distance to the ground is the second preset height H2. The first preset height H1 is smaller than the second preset height H2.

Mark area reduction means 903: This is for determining a closed area surrounded by the lowest mark point and the highest mark point as a mark area.

In some embodiments, when the marking target is a vehicle, the mark area reduction means 903 marks the first intersection A and the second intersection B as the lowest mark points, and the third intersection G and the fourth intersection H as the highest marks. Mark the dots. Therefore, the mark area is a trapezoidal ABGH area surrounded by the first intersection A, the second intersection B, the third intersection G, and the fourth intersection H.

When the mark target is a pedestrian, the mark area reduction means 903 marks the first intersection A and the second intersection B as the lowest mark points, and the third intersection C and the fourth intersection D as the highest mark points. Therefore, the mark area is an ABCD area surrounded by the first intersection A, the second intersection B, the third intersection C, and the fourth intersection D.

When the mark target is a traffic sign, the mark area reduction means 903 marks the first intersection E and the second intersection F as the lowest mark points, and the third intersection G and the fourth intersection H as the highest mark points. Therefore, the mark area is an EFGH area surrounded by the first intersection E, the second intersection F, the third intersection G, and the fourth intersection H.

In some embodiments, the mark processing model acquisition module 803 is also for creating an attention model based on the mark target and the corresponding mark area when the mark target is a vehicle.

In some embodiments, the navigation target marking device marks the marked area using an attention model and obtains an attention thermodynamic chart, which is a thermodynamic advantage for marking the mark image. It also includes a chart generation module. Attention thermodynamic charts show that the closer you are to the red area, the more likely it is that a vehicle will appear, reducing Mark's labor intensity.

In some embodiments, the mark processing model acquisition module 803 is further used to create a pedestrian mark model that matches a pedestrian based on the pedestrian and the pedestrian mark area when the mark target is a pedestrian. Be done.

In some embodiments, the mark processing model acquisition module 803 is further used to create a traffic sign model based on a color model when the mark target is a traffic sign. That is, the type of traffic sign is identified based on the color and shape. For example, a red circle is a prohibited traffic sign, a yellow triangle is a warning traffic sign, and a blue square is an instruction traffic sign.

FIG. 10 is a principle block diagram of the navigation target marking device provided by the fifth embodiment of the present publication. Referring to FIG. 10, the marking device of the navigation target includes the acquisition module 1001, the approximate image exclusion module 1002, the mark area reduction module 1003 and the mark processing model acquisition module 1004, of which the acquisition module 1001 and the mark area reduction module 1003. The functions and operations of the mark processing model acquisition module 1004, the acquisition module 801 in the fourth embodiment, the mark area reduction module 802, and the mark processing model acquisition module 803 are equivalent and will not be described again here. Only the different parts will be described in detail below.

The approximate image exclusion module 1002 is for eliminating the approximate image in the mark image. The approximate image exclusion module 1002 calculates the Hamming distance between all the mark images using the difference value hash algorithm, sets the similarity threshold as the Hamming distance, and uses the image in the section where the humming distance is [S1, S2] as the approximate image. Set.

For example, the similarity threshold is set to the Hamming distances S1 = 0 and S2 = 10, the Hamming distances of all the mark images are calculated, and the mark images whose Hamming distances are within [0, 10] are regarded as approximate images. The approximate image is excluded, and only one of the approximate images is used as the mark image.

FIG. 11 is a principle block diagram of the navigation target marking device provided by the sixth embodiment of the present publication. Referring to FIG. 11, the navigation target marking device includes the acquisition module 1101, the approximate image exclusion module 1102, the latitude and longitude determination module 1103, the mark target determination module 1104, the mark area reduction module 1105, and the mark processing model acquisition module 1106. , Acquisition module 1101, mark area reduction module 1105 and mark processing model acquisition module 1106 and acquisition module 801 in the fourth embodiment, mark area reduction module 802 and mark processing model acquisition module 803 have the same and similar functions. The functions of the image exclusion module 1102 and the approximate image exclusion module 1002 in the fifth embodiment are the same and will not be described again here. Only the different parts will be described in detail below.

In the embodiment of the present publication, the latitude and longitude determination module 1103 is for acquiring the latitude and longitude information of the location position of the mark image.

Of these, latitude and longitude information can be determined by a positioning system mounted on the vehicle, and the present invention is not limited to this.

The mark target determination module 1105 is for determining the mark target based on the latitude and longitude information.

In some embodiments, the mark target determination module 1105 can determine the scene of the mark image based on the latitude and longitude information, and by determining the mark target based on the scene, the mark target is reduced, and further. Improves mark efficiency.

For example, when determining whether the scene of the mark image is a suburb or a distant region based on the latitude and longitude information, the mark target may be a traffic sign and a vehicle. When determining that the scene of the mark image is in the city based on the latitude and longitude information, the mark target may be a pedestrian, a vehicle, and a traffic sign.

The navigation target marking device provided by the embodiment of the present publication acquires a mark image using an acquisition module, and the mark area reduction module reduces the mark area in the mark image based on the mark target, and the mark processing model acquisition module. Creates a mark processing model that matches the mark target based on the mark target and the corresponding mark area. Among them, the mark processing model is for reducing the mark area in the mark image. The mark device can reduce the mark area in the mark image, shorten the mark range, improve the mark efficiency, reduce the labor intensity, reduce the influence on the health of the mark personnel, and reduce the error rate of the mark. can do.

In the third aspect, with reference to FIG. 12, the embodiments of this publication are:
With one or more processors 1201
One or more programs are stored and one or more programs are executed on one or more processors to allow one or more processors to implement the method of marking any of the navigation targets described above. Memory 1202 and
It provides electronic equipment, including one or more I / O interfaces 1203, which are connected between the processor and memory and arranged to enable information exchange between the processor and memory.

Among them, the processor 1201 is a device having data processing capability, and includes, but is not limited to, a central processing unit (CPU) and the like. Memory 1202 is a device having data storage capability, and is a random access memory (RAM, more specifically, for example, SDRAM, DDR, etc.), a read-only memory (ROM), and a charge-erasable programmable read-only memory (EEPROM (registered trademark)). )), Flash memory (FLASH®), but not limited to these. The I / O interface (read / write interface) 1203 is connected between the processor 1201 and the memory 1202, and can realize information exchange between the processor 1201 and the memory 1202, including but not limited to the data bus. ..

In some embodiments, the processor 1201, the memory 1202 and the I / O interface 1203 are connected by a bus and further to other components of the computing facility.

In the fourth aspect, the embodiments of this publication are
When the computer program is stored and executed by the processor, the computer program provides a computer-readable storage medium that realizes the marking method of any of the navigation targets described above.

One of ordinary skill in the art can understand that all or some of the steps, systems, and functional modules / means in the methods published in the preamble can be implemented as software, firmware, hardware, and appropriate combinations thereof. .. In the hardware embodiment, the division between functional modules / means mentioned above does not necessarily correspond to the division of physical components. For example, a physical component may have multiple functions, or a function or step may be performed with a few physical components. Some or all physical components may be executed by a processor (eg, a central processing unit, digital signal processor, or microprocessor) as software or hardware, or as an integrated circuit (eg, dedicated). It can be implemented as an integrated circuit). Such software can be installed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium is practiced in any way or technique for storing information (eg, computer-readable instructions, data structures, program modules, or other data). Includes volatile and non-volatile data, removable and non-removable media. Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROMs, digital multifunction disks (DVDs) or other optical disk storage, cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices. , Or other media that store the desired information and are accessible to the computer, but not limited to. In addition, as is well known to those skilled in the art, communication media generally include computer-readable instructions, data structures, program modules, or other data in modulated data signals of carrier waves or other transfer mechanisms. , Any information distribution medium may be included.

In the fifth aspect, the embodiments of this publication are
Includes a computer program that implements the method of marking any of the navigation targets described above when executed by a processor.
Providing computer program products.

The program code for implementing this published method may be written in any combination of one or more programming languages. These program codes are provided to the processor or controller of a general purpose computer, dedicated computer, or other programmable data processing device and are specified in flowcharts and / or block diagrams when the program code is executed by the processor or controller. It is possible to execute the performed function / operation. The program code can be run entirely on the machine, or partially on the machine, and as a separate software package, partially on the machine and partially on the remote machine, or on the remote machine. It can be fully run or run on the server.

Illustrative examples have been published in the text and used specific terms, but these terms should be used and interpreted merely as a general descriptive meaning and are used for limiting purposes. do not have. In some examples, the features, characteristics and / or elements described in combination with certain embodiments may be used alone and the features, characteristics and / or elements described in combination with other examples, unless otherwise stated explicitly. It is self-evident to those skilled in the art that it can be used in combination with elements. Therefore, various changes in each form and details can be made without departing from the scope of the present disclosure described in the attached claims.

Claims (15)

Steps to get the mark image and
A step of reducing the mark area in the mark image based on the mark target, and
Including a step of creating a mark processing model matching the mark target based on the mark target and the corresponding mark area.
The mark processing model is a method of marking a navigation target in which a mark area in the mark image is reduced. The step of reducing the mark area in the mark image based on the mark target is
The step of determining the vanishing point in the mark image and
A step of determining the lowest mark point and the highest mark point based on the vanishing point and the mark target, and
A step of determining the sealed area surrounded by the lowest mark point and the highest mark point as the mark area, and
The method of claim 1, comprising. The step of determining the lowest mark point and the highest mark point based on the vanishing point and the mark target is
When the mark target is a vehicle, the lowest mark point includes a first intersection where the base of the mark image intersects the left side, and a second intersection whose bottom intersects the right side, and the highest mark point. Refers to a third intersection where the continuous line between the vanishing point and the first intersection intersects the far end line, and a fourth intersection where the continuous line between the vanishing point and the second intersection intersects the far end line. Including, the far end line includes a line parallel to the base of the mark image and a preset distance from the base of the mark image.
The method according to claim 2. The step of determining the lowest mark point and the highest mark point based on the vanishing point and the mark target is
When the mark target is a pedestrian, the lowest mark point includes a first intersection where the base of the mark image intersects the left side and a second intersection whose bottom intersects the right side, and the highest mark. The points include a third intersection where the altitude line intersects the left side of the mark image and a fourth intersection where the altitude line intersects the right side of the mark image, and the altitude line has the vanishing point. Including that the distance to the ground through is a line of preset height,
The method according to claim 2. The step of determining the lowest mark point and the highest mark point based on the vanishing point and the mark target is
When the mark target is a traffic sign, the lowest mark points are the first intersection where the first altitude line intersects the left side of the mark image, and the first altitude line is the right side of the mark image. The highest mark point includes a second intersection that intersects, a third intersection where the second altitude line intersects the left side of the mark image, and a fourth intersection where the second altitude line intersects the right side of the mark image. The first altitude line is a line having a first preset height that passes through the vanishing point and is at a distance from the ground, and the second altitude line passes through the vanishing point and is with the ground. The distance is a second preset height line, including that the first preset height is smaller than the second preset height.
The method according to claim 2. After the step of creating a mark processing model that matches the mark target based on the mark target and the corresponding mark area.
The step of acquiring the thermodynamic chart of the mark region based on the mark processing model, and
The method according to any one of claims 1 to 5, further comprising a step of marking a target in the marked area based on the thermodynamic chart. Before the step of reducing the mark area in the mark image based on the mark target,
The method according to any one of claims 1 to 5, further comprising a step of excluding an approximate image in the mark image. Before the step of reducing the mark area in the mark image based on the mark target,
The step of acquiring the latitude and longitude information of the location of the mark image and
The method according to any one of claims 1 to 5, further comprising a step of determining the mark target based on the latitude and longitude information. The acquisition module for acquiring the mark image and
A mark area reduction module for reducing the mark area in the mark image based on the mark target,
Includes a mark processing model acquisition module that creates a mark processing model that matches the mark target based on the mark target and the corresponding mark area.
The mark processing model is a navigation target marking device for reducing a mark area in the mark image. The mark area reduction module
A vanishing point determining means for determining the vanishing point in the mark image, and
A mark point determination means for determining the lowest mark point and the highest mark point based on the vanishing point and the mark target, and
The apparatus according to claim 9, further comprising a mark area reducing means for determining a closed area surrounded by the minimum mark point and the maximum mark point as the mark area. The device is
The apparatus according to claim 9, further comprising an approximate image exclusion module for eliminating an approximate image in the mark image. The device is
A latitude and longitude determination module for acquiring latitude and longitude information of the location of the mark image,
The device according to claim 9, further comprising a mark target determination module for determining the mark target based on the latitude and longitude information. With one or more processors
One or more programs are stored, and the one or more programs are executed by the one or more processors, whereby any one of claims 1 to 8 is applied to the one or more processors. A memory that realizes the method described in paragraph 1 and
An electronic facility that includes one or more I / O interfaces that are connected between the processor and memory and arranged to provide information alternating between the processor and memory. A computer-readable storage medium that implements the method according to any one of claims 1 to 8 when a computer program is stored and the computer program is executed by a processor. Includes a computer program that implements the method according to any one of claims 1-8 when executed by a processor.
Computer program product.

Images