JP2023047584A - Location management system and location management method - Google Patents

Abstract

To obtain highly accurate position information of a moving object at a low cost.SOLUTION: A position management system according to an embodiment acquires a photographed image from a stationary camera, detects a moving object from the photographed image, and specifies actual position information of the moving object in the photographed image on the basis of position correspondence information in which the actual position information is associated in advance with each part of the photographed image. Also, the position management system acquires position information of the moving object based on GPS or GNSS from the moving object, acquires predetermined feature information about the moving object from the moving object or an external device, and extracts predetermined feature information of the moving object from the photographed image. Also, the position management system associates the moving object appearing in the photographed image with a moving object that transmitted the position information acquired by a position information acquisition unit on the basis of the actual position information specified by a specifying unit, the position information acquired by the position information acquisition unit, the predetermined feature information acquired by a feature information acquisition unit, and the predetermined feature information extracted by an extraction unit.SELECTED DRAWING: Figure 12

Description

TECHNICAL FIELD Embodiments of the present invention relate to a location management system and a location management method.

In recent years, for example, in order to realize a dynamic map used for MaaS (Mobility as a Service), it is necessary to measure and manage accurate position information regarding moving objects such as vehicles. When obtaining vehicle position information, for example, GNSS (Global Navigation Satellite System) can be used rather than GPS (Global Positioning System) to obtain highly accurate position information.

However, even if GNSS is used, there is a problem that, for example, errors in position information increase in areas where tall buildings stand side by side. Therefore, in order to realize advanced automatic driving, it is necessary to reduce position measurement errors.

JP 2018-32433 A JP 2018-81252 A Japanese Patent Application Laid-Open No. 2020-30362

However, there is a problem that the high-precision positioning device described above is expensive.

Therefore, the problem to be solved by the present invention is to provide a position management system and a position management method capable of obtaining highly accurate position information of an object at low cost.

A position management system according to an embodiment includes a captured image acquisition unit that acquires a captured image from a fixed camera that captures a predetermined area in which a moving object moves, a detection unit that detects the moving object from the captured image, and the fixed camera. a specifying unit for specifying the real position information of the moving object appearing in the photographed image based on the position correspondence information in which the real position information is associated in advance with each part of the photographed image; A position information acquisition unit that acquires position information of the moving object based on GPS or GNSS from the moving object moving in an area, and acquires predetermined feature information about the moving object from the moving object or an external device. a feature information acquiring unit; an extracting unit that extracts the predetermined feature information of the moving object from the captured image; the actual position information specified by the specifying unit; and the position acquired by the position information acquiring unit. information, the predetermined feature information acquired by the feature information acquisition unit, and the predetermined feature information extracted by the extraction unit, the moving object appearing in the captured image, the and a control unit that associates the moving object that transmitted the position information acquired by the position information acquisition unit.

FIG. 1 is a schematic diagram for explaining rough differences between the embodiment and the comparative example. FIG. 2 is an overall configuration diagram of the location management system of the embodiment. FIG. 3 is a diagram for explaining an outline of processing by a PC according to the embodiment; FIG. 4 is a diagram for explaining the correspondence relationship between the stationary camera image and the satellite image in the embodiment. FIG. 5 is a diagram showing a conversion table according to the embodiment. FIG. 6 is a diagram for explaining an example of a vehicle height calculation method according to the embodiment. FIG. 7 is a diagram showing an example of a vehicle type discrimination table in the embodiment. FIG. 8 is a diagram for explaining an example of the vehicle body color determination method in the embodiment. FIG. 9 is a diagram showing an example of body color determination results in the embodiment. FIG. 10 is a flow chart showing processing by the PC of the embodiment. FIG. 11 is a flow chart showing details of the processing in step S3 of FIG. FIG. 12 is a flowchart illustrating processing by the location management server according to the embodiment; FIG. 13 is a flow chart showing the details of the processing in step S43 of FIG. FIG. 14 is a flowchart illustrating another process by the location management server of the embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of a position management system and a position management method of the present invention will be described with reference to the drawings. A vehicle will be described below as an example of a moving object.

First, with reference to FIG. 1, the rough differences between the embodiment and the comparative example (prior art) will be described.
FIG. 1 is a schematic diagram for explaining rough differences between the embodiment and a comparative example (conventional technology). In a comparative example (conventional technology), it is assumed that a dynamic map is created using position information from a high-precision positioning device mounted on a vehicle. In this case, highly accurate position information of the vehicle can be obtained in the entire area in which the vehicle travels, but there is a problem that the highly accurate positioning device is expensive.

On the other hand, in the present embodiment, it is assumed that a dynamic map is created using position information based on the proposed model described below. In the proposed model, roads are classified into two types of areas, and the level of vehicle positioning accuracy is divided. One is an area such as an intersection that requires high-precision position measurement, and is an application target of the proposed model. The other is an area such as a straight road that does not require high-precision position measurement, and the proposed model is not applicable, and the position information of the GPS sensor possessed by each vehicle is used. According to this technique, it is possible to obtain highly accurate vehicle position information at low cost in a highly accurate position measurement area. Details will be described below.

FIG. 2 is an overall configuration diagram of the location management system S of the embodiment. The location management system S includes a location management server 1, a PC (Personal Computer) 2, a stationary camera 3, and a map management server 4 (management server).

The stationary camera 3 photographs a predetermined area (for example, an intersection of roads, etc.) in which a vehicle 5 (hereinafter also referred to as "vehicle" without reference numeral) moves. More specifically, the stationary camera 3 is installed, for example, at a high position in a major location on the road, and has an angle of view that looks down on the road from above. The range of the field of view of the stationary camera 3 becomes a highly accurate position measurement area. In addition, fixed cameras 3 are installed at various places on the road, and one PC 2 is installed for one fixed camera 3 .

Here, with reference to FIG. 3, an overview of the processing by the PC 2 of the embodiment will be described.
FIG. 3 is a diagram for explaining an outline of processing by the PC 2 of the embodiment. FIG. 3A is an example of an image obtained by the stationary camera 3 (hereinafter also referred to as a stationary camera image).

As shown in FIG. 3(a), the PC 2 detects the vehicle from the stationary camera image. In FIG. 3(b), the detected vehicle is surrounded by a rectangle. Then, the PC 2 identifies position information of the detected vehicle. FIG. 3C schematically shows that the center position P10 of the bottom surface of the wire frame L applied to the vehicle is specified as the vehicle position information using the wire frame method.

Returning to FIG. 2 , the PC 2 includes a processing section 21 , a storage section 22 , an input section 23 , a display section 24 and a communication section 25 .

The storage unit 22 is a storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various information. The storage unit 22 stores, for example, a position DB (Data Base) 221 (position correspondence information), a conversion table 222 (position correspondence information), first vehicle information 223, and a vehicle type discrimination table 224 .

The position DB 221 is a DB that defines a correspondence relationship between a fixed camera image and a satellite image (or an aerial image, an example of a bird's-eye view image). Here, with reference to FIG. 4, the correspondence relationship between the stationary camera image and the satellite image in the embodiment will be described.

FIG. 4 is a diagram for explaining the correspondence relationship between the stationary camera image and the satellite image in the embodiment. In the position DB 221, each pixel in the stationary camera image shown in FIG. 4(a) is associated with each pixel in the satellite image shown in FIG. 4(b) for the movable range of the vehicle 5. FIG. Therefore, by referring to the position DB 221, it is possible to specify the pixels in the satellite image corresponding to the pixels in the stationary camera image. Also, the position DB 221 may be realized by a projective transformation matrix generated based on a plurality of sets of coordinates of corresponding points between the stationary camera image and the satellite image.

Returning to FIG. 2, the conversion table 222 is a table in which the coordinates of the pixels of the satellite image and the position information of the satellite image are associated with each other. Here, the conversion table 222 of the embodiment will be described with reference to FIG.

FIG. 5 is a diagram showing the conversion table 222 of the embodiment. In the conversion table 222, the coordinates (X coordinate, Y coordinate) of the pixels of the satellite image are associated with highly accurate position information (latitude, longitude, altitude) of the satellite image.

Returning to FIG. 2, the first vehicle information 223 is vehicle information detected based on the stationary camera image. The first vehicle information 223 includes, for example, information (1) to (11) below.
(1) Shooting date and time (2) Frame number of stationary camera image (3) Coordinates of representative points (for example, upper left and lower right) of rectangle surrounding vehicle (4) Bottom center coordinates of wire frame (5) Vehicle type (ordinary vehicle) / truck / bus etc.)
(6) Body color information (for example, RGB (Red, Green, Blue) histogram)
(7) Tracking ID (Identifier) (when tracking vehicles in chronological order)
(8) Location information (latitude, longitude, altitude)
(9) Direction of vehicle (10) Driving direction (advancing direction)
(11) Driving lane information

Among (1) to (11), (1) and (2) are obtained from the stationary camera image. (3) to (11) are obtained by the processing of the processing unit 21. FIG.

The processing unit 21 controls the overall operation of the PC 2 and implements various functions of the PC 2 . The processing unit 21 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU comprehensively controls the operation of the PC2. The ROM is a storage medium that stores various programs and data. RAM is a storage medium for temporarily storing various programs and rewriting various data. The CPU uses the RAM as a work area (working area) to execute programs stored in the ROM, the storage unit 22, and the like. The processing unit 21 includes an acquisition unit 211, a detection unit 212, an identification unit 213, an extraction unit 214, and a control unit 215 as functional units.

The acquisition unit 211 acquires various types of information from an external device (eg, the stationary camera 3, the position management server 1, etc.). The acquisition unit 211 functions as a captured image acquisition unit that acquires a captured image from the stationary camera 3, for example.

A detection unit 212 detects a vehicle from a stationary camera image based on a technique such as DNN (Deep Neural Network).

For example, based on the position DB 221 and the conversion table 222, the identification unit 213 determines the actual position of the vehicle based on the center position of the bottom surface of the wire frame applied to the vehicle captured in the stationary camera image and the corresponding position in the satellite image. Identify information.

The extraction unit 214 extracts predetermined characteristic information of the vehicle from the captured image. The predetermined feature information is, for example, vehicle type information indicating the type of vehicle. In that case, the extraction unit 214 extracts vehicle type information from the captured image. When extracting the vehicle type information from the captured image, the extraction unit 214 estimates the orientation of the vehicle 5 in the captured image, expresses the vehicle with a rectangular parallelepiped wire frame based on the orientation, and extracts the vehicle type information from the captured image. to calculate the appearance characteristics, vehicle width, vehicle length, and vehicle height. Then, the extracting unit 214 extracts the vehicle type discrimination table 224 (vehicle type) in which the calculated external features, vehicle width, vehicle length, vehicle height, and vehicle type information are associated in advance with the external features, vehicle width, vehicle length, vehicle height, and vehicle type information. Correspondence information), and vehicle type information is extracted.

Here, vehicle type determination and the like will be described with reference to FIGS. 6, 7, and the like. First, an example of a vehicle height calculation method will be described with reference to FIGS. 3(c) and 6. FIG.

FIG. 6 is a diagram for explaining an example of a vehicle height calculation method according to the embodiment. The coordinates of the four points (P1 to P4 in FIG. 3(c)) at the corners of the bottom surface of the wire frame are projectively transformed onto the map. Next, by calculating the GPS coordinates of the four points after projective transformation and converting the difference in coordinates into a distance, the vehicle width (the length between P1 and P3) and the vehicle length (the length between P1 and P2) ) is calculated. For example, these processes are repeated for each wire frame of a plurality of captured images of the same vehicle, and an average is obtained.

Next, a method for calculating the vehicle height will be described. First, P1 and P2, which are both ends of the vehicle length at the bottom of the wire frame, are selected.

Next, P1 and P1h, which are both ends of the vehicle height in the wire frame, are selected.
Next, let P1a (x1−N, y1) be the coordinate that is N pixels to the left of the screen x coordinate of P1.
Also, let P1b (x1+N, y1) be the coordinate that is N pixels to the right of the screen x coordinate of P1.

Similarly, let P2a (x2-N, y2) be the coordinate that is N pixels to the left of the screen x-coordinate of P2.
Also, let P2b (x2+N, y2) be the coordinate that is N pixels to the right of the screen x coordinate of P1.

Next, the screen coordinates of P1a and P1b are projectively transformed to calculate the latitude and longitude.
Next, from the relationship between the calculated values for the vehicle width and length and the number of pixels on the image, it is possible to calculate the approximate length between pixels (considering that it varies depending on the position on the screen), so that value can be used. Determine the distance 1 between P1a and P1b. For P1, the distance 11 between the x-coordinate pixels can be calculated as the distance 1÷2N.

Similarly, for P2, the distance 12 between the x-coordinate pixels can be calculated by (distance 2 between P2a and P2b)/2N.

The vehicle height is (average of distance 11 and distance 12)×(y coordinate of P1−y coordinate of P1h) (distance D in FIG. 6). Further, as shown in FIG. 6, correction based on the angle of view α of the stationary camera 3 (varying according to the y-coordinate of the screen) is performed based on the following equation.
Actual vehicle height H = distance D ÷ cos (angle of view α)

Next, an example of the vehicle type determination table 224 in the embodiment will be described with reference to FIG.
FIG. 7 is a diagram showing an example of the vehicle type discrimination table 224 in the embodiment. Using the information (21) and the information (22), the vehicle type of the vehicle detected by the fixed camera 3 is estimated.

Information (21) is a vehicle type classification based on appearance features using DL/DNN. Information (22) is information on vehicle width, vehicle length, and vehicle height based on the result of vehicle dimension measurement using a wire frame. With information (21) alone, it is difficult to distinguish vehicles that are similar in appearance but differ in size. However, with the additional information (22) such a distinction is possible.

In addition, information on vehicle width, vehicle length, and vehicle height for each vehicle type is defined in advance. As a result, the vehicle type can be estimated with high accuracy using the vehicle type identification table 224, the information (21), and the information (22).

Note that if this method makes the determination result unstable, for example, the information (22) is prioritized over the information (21). Also, if any of the vehicle width, vehicle length, and vehicle height measurement values in the information (22) does not fall within the range of standard dimensions for each vehicle type defined in the vehicle type identification table 224, for example, three measurements Select the car model with the smallest error between the value and the standard dimension.

Returning to FIG. 2, the extraction unit 214 extracts body color information of the vehicle 5 from the captured image, for example. When extracting the body color information of the vehicle 5 from the captured image, the extraction unit 214 estimates the posture of the vehicle 5 captured in the captured image, expresses the vehicle with a rectangular parallelepiped wire frame based on the posture, and extracts the vehicle body color information from the captured image. Two representative colors are extracted as vehicle body color information by clustering the color information based on the cropped image which is the image of the portion of the vehicle 5 inside.

Here, with reference to FIG. 8, an example of the body color (color of the vehicle 5) determination method in the embodiment will be described.
FIG. 8 is a diagram for explaining an example of the vehicle body color determination method in the embodiment. The body color is determined by the following processes (31) to (35).

First, in processing (31), a cropped image, which is an image of the portion of the vehicle 5 inside the wire frame in the captured image, is extracted.

Next, in process (32), color information is obtained from the cropped image using a clustering method. In other words, since the cropped image contains multiple types of colors with slightly different brightness and color, by clustering similar colors together, 5 to 7 colors are collected and color information is extracted. .

Next, in process (33), the color information is represented by a histogram (may be normalized).
Next, in process (34), the black component of the vehicle tires is removed. For example, if the body is black, a certain percentage (eg 10%) of the black component is removed.

Next, in process (35), the body color is represented by two representative colors based on the color histogram. For example, the color with the most frequent value in the color histogram is set as the main color, and the color with the second highest frequency in the color histogram or the color that characterizes the vehicle (color that is not found in surrounding vehicles) is set as the auxiliary color. Note that two representative colors may be used without distinguishing between the main color and the auxiliary color.

In this way, the body color can be determined. Note that, depending on the orientation of the car and the degree of reflection of light on the glass, the result of determining the body color may change. It is not limited to this.

Next, with reference to FIG. 9, an example of the vehicle body color determination result in the embodiment will be described.
FIG. 9 is a diagram showing an example of body color determination results in the embodiment. Body color determination was performed on the photographed image shown on the left. The body colors of the six vehicles detected from the captured images were determined to be white, blue, black, blue, black, and green in order from the top.

Returning to FIG. 2, the extracting unit 214 extracts traveling direction information of the vehicle from a plurality of captured images in time series, for example.
The extraction unit 214 also extracts, for example, lane information along which the vehicle travels from the captured image.

The control unit 215 executes various types of information processing. Further, the control unit 215 appropriately generates information that is not generated by the identifying unit 213 and the extracting unit 214 among the above (3) to (11).

The input unit 23 receives user's operations on the PC 2 . The input unit 23 is, for example, an input device such as a keyboard and a mouse.

The display unit 24 displays various information. The display unit 24 is, for example, a liquid crystal display (LCD), an organic EL (Electro-Luminescence) display, or the like.

The communication unit 25 is a communication interface for communicating with external devices (eg, the stationary camera 3, the position management server 1, etc.).

Next, the location management server 1 will be explained. The location management server 1 is a computer device and includes a processing section 11 , a storage section 12 , an input section 13 , a display section 14 and a communication section 15 .

The storage unit 12 is a storage device such as an HDD or SSD, and stores various information. The storage unit 12 stores first vehicle information 121, second vehicle information 122, and correspondence information 123, for example.

The first vehicle information 121 is the same as the first vehicle information 223 in PC2.

The second vehicle information 122 is the same as the second vehicle information 51 of the vehicle 5 . The second vehicle information 51 includes, for example, information (16) to (18) below.
(16) GPS date and time (17) GPS location information (latitude, longitude, altitude)
(18) ETC registration information (in-vehicle device number, registration number/vehicle number, traction device presence/absence information, etc.)

The correspondence information 123 is information in which the first vehicle information 121 and the second vehicle information 122 are associated with each other for the same vehicle 5 .

The processing unit 11 controls the overall operation of the location management server 1 and implements various functions of the location management server 1 . The processing unit 11 includes, for example, a CPU, a ROM, and a RAM. The CPU comprehensively controls the operation of the location management server 1 . The ROM is a storage medium that stores various programs and data. RAM is a storage medium for temporarily storing various programs and rewriting various data. The CPU uses the RAM as a work area and executes programs stored in the ROM, the storage unit 12, and the like. The processing unit 11 includes an acquisition unit 111 and a control unit 112 as functional units.

Acquisition unit 111 acquires various types of information from an external device (eg, PC 2, map management server 4, etc.). For example, the acquisition unit 111 acquires the first vehicle information 223 from the PC 2 and stores the first vehicle information 223 in the storage unit 12 as the first vehicle information 121 . Further, the acquisition unit 111 acquires the second vehicle information 51 from the vehicle 5 and stores it as the second vehicle information 122 in the storage unit 12, for example.

The acquisition unit 111 also functions as a position information acquisition unit that acquires position information of the vehicle 5 based on GPS or GNSS from the vehicle 5 moving in a predetermined area.

The acquisition unit 111 also functions as a feature information acquisition unit that acquires predetermined feature information (vehicle type information, body color information, traveling direction information, lane information, etc.) regarding the vehicle 5 from the vehicle 5 or the external device 6 . The external device 6 is a computer device that manages feature information.

The control unit 112 executes various types of information processing. The control unit 112, for example, acquires the actual position information of the vehicle 5, the position information acquired by the position information acquisition unit (acquisition unit 111), and the predetermined feature information acquired by the feature information acquisition unit (acquisition unit 111). , the predetermined feature information extracted by the extraction unit 214, the vehicle 5 appearing in the captured image, the vehicle 5 that transmitted the position information acquired by the position information acquisition unit (acquisition unit 111), correspond. The associated information is stored in the correspondence information 123 .

After performing the above-described association, the control unit 112 transmits information such as “(8) position information (latitude, longitude, altitude)” in the first vehicle information 121 to the map management server 4 . As a result, the map management server 4 can update the dynamic map 41 using highly accurate "(8) position information (latitude, longitude, altitude)" as auxiliary information.

Also, the control unit 112 transmits information such as “(8) position information (latitude, longitude, altitude)” in the first vehicle information 121 to the corresponding vehicle 5 . Thereby, the vehicle 5 can utilize highly accurate "(8) position information (latitude, longitude, altitude)".

The input unit 13 receives a user's operation on the location management server 1 . The input unit 13 is, for example, an input device such as a keyboard and a mouse.

The display unit 14 displays various information. The display unit 14 is, for example, a liquid crystal display (LCD), an organic EL display, or the like.

The communication unit 15 is a communication interface for communicating with an external device (for example, the PC 2, the map management server 4, etc.).

Next, referring to FIG. 10, processing by the PC 2 of the embodiment will be described.
FIG. 10 is a flow chart showing processing by the PC 2 of the embodiment.
First, in step S<b>1 , the acquisition unit 211 acquires a stationary camera image from the stationary camera 3 .

Next, in step S2, the detection unit 212 detects the vehicle 5 (moving object) captured in the stationary camera image based on a technique such as DNN.

Next, in step S3, the identification unit 213 identifies the position information of the vehicle 5 (moving object). Details of the process of step S3 in FIG. 10 will now be described with reference to FIG.

FIG. 11 is a flow chart showing details of the processing in step S3 of FIG.
In step S31, the specifying unit 213 refers to the position DB 221 and performs coordinate transformation of the stationary camera image into a satellite image (FIG. 4).

Next, in step S32, the specifying unit 213 calculates bottom center coordinates of the wire frame applied to the vehicle 5 (moving object) in the satellite image (FIG. 3(c)). Next, in step S33, the identification unit 213 refers to the conversion table 222 and converts the bottom center coordinates of the vehicle in the satellite image into position information (latitude, longitude, altitude).

Returning to FIG. 10, after step S3, the control unit 215 generates the first vehicle information 223 in step S4. Next, in step S<b>5 , the control section 215 transmits the first vehicle information 223 to the position management server 1 .

Next, processing by the location management server 1 of the embodiment will be described with reference to FIG. 12 .
FIG. 12 is a flowchart showing processing by the location management server 1 of the embodiment.
First, in step S<b>41 , the acquisition unit 111 acquires the first vehicle information 223 from the PC 2 and stores it in the storage unit 12 as the first vehicle information 121 .

Next, in step S<b>42 , the acquisition unit 111 acquires the second vehicle information 51 from the vehicle 5 and stores it in the storage unit 12 as the second vehicle information 122 .

Next, in step S<b>43 , the control unit 112 associates the first vehicle information 121 and the second vehicle information 122 with respect to the same vehicle 5 and stores the processing result in the storage unit 12 as the correspondence information 123 . Details of the processing in step S43 of FIG. 12 will now be described with reference to FIG.

FIG. 13 is a flow chart showing the details of the processing in step S43 of FIG.
In step S431, the control unit 112 narrows down by position (including lane). For example, (8) positional information (latitude, longitude, altitude) of the first vehicle information 121 and (17) GPS positional information of the second vehicle information 122 are used to associate them. This association is performed using location information in a map coordinate system (latitude and longitude). It can be difficult.

Next, in step S<b>432 , the control unit 112 narrows down according to the traveling direction and speed of the vehicle 5 . As for the direction of travel, for example, cars that go straight or cars that turn left are associated with each other. Next, in step S433, the control unit 112 narrows down by vehicle type and body color. As for the body color, for example, two representative colors are used to associate the body color.

Note that the order of the processing of steps S431 to S433 is not limited to this. For example, the order of steps S432 and S433 may be reversed, or if the matching is not successful in the first round of processing of steps S431 to S433, the second round of processing may be performed. In addition, instead of using the elements for narrowing down in order, they may be used concurrently, or elements with a high probability of narrowing down may be preferentially used. In addition, the vehicles 5 may be associated not one by one, but three or four vehicles collectively.

Returning to FIG. 12, in step S44 after step S43, the control unit 112 transmits information such as "(8) position information (latitude, longitude, altitude)" in the first vehicle information 121 to the map management server 4. do.

Next, in step S<b>45 , the control unit 112 transmits information such as “(8) position information (latitude, longitude, altitude)” in the first vehicle information 121 to the corresponding vehicle 5 .

Next, other processing by the location management server 1 of the embodiment will be described with reference to FIG. 14 .
FIG. 14 is a flow chart showing another process by the location management server 1 of the embodiment. The number of vehicles recognized based on the first vehicle information 121 and the number of vehicles recognized based on the second vehicle information 122 may differ. Here, it is assumed that the number of vehicles recognized based on the second vehicle information 122 is correct.

In that case, the vehicle detection accuracy based on the stationary camera image may deteriorate due to adverse conditions such as nighttime or bad weather. Specifically, it is conceivable that the number of vehicle rectangles detected from the stationary camera image may be smaller or larger than the number of vehicles traveling on the road.

If the number of vehicle rectangles is small, the conditions for vehicle detection are too strict. Further, when the number of vehicle rectangles is large, the condition for vehicle detection is too loose. Therefore, the processing of FIG. 14 is performed.

First, in step S51, the control unit 112 uses the second vehicle information 122 to calculate the second number of vehicles (N2), which is the number of vehicles.

Next, in step S52, the control unit 112 uses the first vehicle information 121 to calculate the first number of vehicles (N1), which is the number of vehicles.

Next, in step S53, the control unit 112 determines whether or not (N2-N1) is greater than a first threshold value (for example, about 3 to 5). If so, the process proceeds to step S55.

In step S54, the control unit 112 switches dictionaries (switching between day and night models, switching between weather models, etc.) and adjusts parameters in the DNN used by the detection unit 212 of the PC2 for vehicle detection using stationary camera images so that N1 increases. I do. After step S54, the process returns to step S52.

In step S55, the control unit 112 determines whether or not (N1-N2) is greater than a second threshold value (for example, about 3 to 5). If Yes, proceed to step S56. exit.

In step S56, the control unit 112 performs dictionary switching and parameter adjustment in the DNN used by the detection unit 212 of the PC2 for vehicle detection using a fixed camera image so as to reduce N1. After step S56, the process returns to step S52. In this way, by switching dictionaries and adjusting parameters as necessary, the numbers of N1 and N2 can be brought close to each other, and the position information of the vehicle can be specified appropriately.

In this way, according to the position management system S of the present embodiment, in addition to the vehicle position information specified from the fixed camera image and the GPS position information acquired from the vehicle, predetermined characteristic information (vehicle type information , body color information, traveling direction information, lane information, etc.), highly accurate position information of a moving object (vehicle) can be obtained at low cost.

Also, when vehicle type information is used, highly accurate association can be performed by using the method described with reference to FIG.

Further, when body color information is used, highly accurate association can be performed by using the method described with reference to FIG. 8 .

Further, by transmitting the vehicle position information to the map management server 4, the map management server 4 can appropriately update the dynamic map 41 using this highly accurate position information as auxiliary information.

In addition, by transmitting the position information of the vehicle to the corresponding vehicle 5, the vehicle 5 can effectively utilize this highly accurate position information for autonomous driving (such as automatic driving).

The program executed by the location management server 1 and the PC 2 of the embodiment has a module configuration including each functional unit described above, and as actual hardware, a CPU (processor) reads the program from the ROM and executes By doing so, each functional unit is loaded onto the main storage device and generated on the main storage device.

The program is recorded in a computer-readable recording medium such as a CD-ROM, a flexible disk, a CD-R, and a DVD as an installable or executable file.

Alternatively, the program may be stored in a computer connected to a network such as the Internet, and provided by being downloaded via the network. Also, the program may be configured to be provided or distributed via a network such as the Internet. Alternatively, the program may be configured to be pre-installed in a ROM or the like and provided.

Although the embodiment of the present invention has been described above, the above embodiment is merely an example and is not intended to limit the scope of the invention. The above embodiments can be implemented in various forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. The above embodiments are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

For example, the moving object is not limited to a vehicle, and may be another moving object such as a moving object (transportation vehicle, mobile robot, etc.) in a factory.

Also, the functions of the location management server 1 and the PC 2 may be realized by one computer device, or may be realized by three or more computer devices.

Also, if the stationary camera image is distorted, the image may be corrected by, for example, straightening the white lines of the pedestrian crossing in the image. Also, if the satellite image is distorted, the image may be corrected in the same manner.

Also, the second vehicle information may be generated by other methods such as GNSS instead of GPS.

Also, a plurality of fixed cameras 3 may be installed for one photographing area, and a plurality of fixed camera images may be used.

DESCRIPTION OF SYMBOLS 1... Position management server 2... PC 3... Stationary camera 4... Map management server 5... Vehicle 11... Processing part 12... Storage part 13... Input part 14... Display part 15... Communication part, 21 processing unit 22 storage unit 23 input unit 24 display unit 25 communication unit 41 dynamic map 51 second vehicle information 111 acquisition unit 112 control unit 121 first Vehicle information 122 Second vehicle information 123 Corresponding information 211 Acquisition unit 212 Detection unit 213 Identification unit 214 Extraction unit 215 Control unit 221 Position DB 222 Conversion table 223... First vehicle information, 224... Vehicle type identification table, S... Position management system

Claims (9)

a captured image acquisition unit configured to acquire a captured image from a stationary camera that captures a predetermined area in which a moving object moves;
a detection unit that detects the moving object from the captured image;
a specifying unit that specifies the real position information of the moving object appearing in the photographed image based on position correspondence information in which real position information is associated in advance with each part of the photographed image by the stationary camera;
a position information acquisition unit that acquires position information of the moving object based on GPS (Global Positioning System) or GNSS (Global Navigation Satellite System) from the moving object moving in the predetermined area;
a feature information acquisition unit that acquires predetermined feature information about the moving object from the moving object or an external device;
an extraction unit that extracts the predetermined feature information of the moving object from the captured image;
The actual position information specified by the specifying unit, the position information acquired by the position information acquisition unit, the predetermined feature information acquired by the feature information acquisition unit, and the a control unit that associates the moving object appearing in the captured image with the moving object that transmitted the position information acquired by the position information acquisition unit, based on the predetermined feature information;
location management system. the moving object is a vehicle,
The predetermined characteristic information is vehicle type information indicating the type of the vehicle,
The feature information acquisition unit acquires the vehicle type information as predetermined feature information about the vehicle from the vehicle or the external device,
2. The position management system according to claim 1, wherein said extraction unit extracts said vehicle type information of said vehicle from said photographed image. When extracting the vehicle type information of the vehicle from the captured image, the extraction unit
Estimating the attitude of the vehicle in the captured image, representing the vehicle in a rectangular parallelepiped wire frame based on the attitude, and calculating exterior features, vehicle width, vehicle length, and vehicle height based on the wire frame. Then, based on the calculated exterior features, the vehicle width, the vehicle length, and the vehicle height, and the vehicle type correspondence information in which the exterior features, the vehicle width, the vehicle length, the vehicle height, and the vehicle type information are associated in advance. 3. The position management system according to claim 2, wherein said vehicle type information of said vehicle is extracted. the moving object is a vehicle,
The predetermined characteristic information is vehicle body color information,
The feature information acquisition unit acquires the vehicle body color information as predetermined feature information about the vehicle from the vehicle or the external device,
2. The position management system according to claim 1, wherein said extraction unit extracts said vehicle body color information of said vehicle from said photographed image. When extracting the vehicle body color information of the vehicle from the captured image, the extraction unit
estimating the posture of the vehicle appearing in the photographed image, expressing the vehicle in a rectangular parallelepiped wire frame based on the posture, and based on a cropped image that is an image of the portion of the vehicle inside the wire frame; , extracting two representative colors as the vehicle body color information by clustering the color information,
5. The position management system according to claim 4, wherein said characteristic information acquisition unit acquires two representative colors as said vehicle body color information relating to said vehicle from said vehicle or said external device. the moving object is a vehicle,
The predetermined characteristic information is traveling direction information of the vehicle,
The feature information acquisition unit acquires the traveling direction information from the vehicle as predetermined feature information about the vehicle,
2. The position management system according to claim 1, wherein said extraction unit extracts said traveling direction information of said vehicle from said plurality of photographed images in time series. the moving object is a vehicle,
The predetermined feature information is lane information on which the vehicle travels,
The feature information acquisition unit acquires, from the vehicle, the lane information on which the vehicle travels as predetermined feature information about the vehicle,
2. The position management system according to claim 1, wherein said extraction unit extracts said lane information along which said vehicle travels from said photographed image. 2. The position management system according to claim 1, wherein said control unit transmits position information of said moving object to a management server of said moving object and/or a dynamic map. a photographed image obtaining step of obtaining a photographed image from a stationary camera that photographs a predetermined area in which the moving object moves;
a detection step of detecting the moving object from the captured image;
a specifying step of specifying the real position information of the moving object appearing in the photographed image based on position correspondence information in which real position information is associated in advance with each part of the photographed image by the stationary camera;
a position information obtaining step of obtaining position information of the moving object based on GPS or GNSS from the moving object moving in the predetermined area;
a feature information acquiring step of acquiring predetermined feature information about the moving object from the moving object or an external device;
an extracting step of extracting the predetermined feature information of the moving object from the captured image;
The actual position information specified by the specifying step, the position information acquired by the position information acquiring step, the predetermined feature information acquired by the feature information acquiring step, and the extracted by the extracting step a control step of associating the moving object appearing in the captured image with the moving object that transmitted the position information acquired by the position information acquiring step, based on the predetermined characteristic information;
location management methods, including;

Images