JP2023084392A - Information processor - Google Patents

Abstract

To enable accurate tracking of vehicles.SOLUTION: An information processor includes a control unit that executes operations of: Transmitting, to a cooperating vehicle, vehicle-related information on a target vehicle to be tracked; acquiring, from the cooperating vehicle, provision information including an image of the target vehicle identified based on the vehicle-related information; and transmitting, to the cooperating vehicle, feature information concerning features of the target vehicle identified from the image.SELECTED DRAWING: Figure 10

Description

The present disclosure relates to an information processing device.

Patent Literature 1 discloses a vehicle search system including a vehicle equipped with an in-vehicle camera and a server configured to be able to communicate with the vehicle. The server transmits the license plate information of the vehicle to be searched to the vehicle. Then, it is disclosed that the vehicle compares the license plate information of the detected vehicle acquired from the image of the camera with the license plate information included in the vehicle data.

JP 2019-67201 A

An object of the present disclosure is to provide a technology that enables tracking of vehicles more accurately than in the past.

An information processing device according to a first aspect of the present disclosure includes:
transmitting vehicle-related information about the target vehicle being tracked to the cooperating vehicle;
Acquiring provided information including an image of the target vehicle identified based on the vehicle-related information from the cooperating vehicle;
transmitting feature information about the features of the target vehicle identified from the image to the cooperating vehicle;
A control unit for executing

The present disclosure enables accurate vehicle tracking.

It is a figure showing an example of composition of an information processing system concerning an embodiment. It is a figure which shows an example of the hardware constitutions of a center server and an in-vehicle device. It is a figure explaining tracking of a target vehicle. It is a figure explaining the characteristic of a target vehicle. It is a figure which shows roughly an example of the functional structure of a center server. It is a figure which shows an example of the provision information stored in provision information DB. It is a figure which shows an example of the characteristic information stored in characteristic information DB. It is a figure showing roughly an example of functional composition of an in-vehicle device. It is a figure which shows an example of the information stored in object vehicle DB. 4 is a sequence diagram showing the flow of processing of the information processing system according to the embodiment; FIG. 4 is a sequence diagram showing the flow of processing of the information processing system according to the embodiment; FIG. It is a figure explaining the pursuit process of a target vehicle.

An information processing device that is one aspect of the present disclosure is an information processing device that is configured to be able to communicate with a cooperative vehicle that cooperates in tracking. A control unit included in the information processing device transmits vehicle-related information about the target vehicle to be tracked to the cooperating vehicle. Vehicle-related information includes, for example, license plate information (vehicle registration number marked on the license plate) of the target vehicle to be tracked. Also, the vehicle-related information may include target area information that indicates a range in which to search for the target vehicle. Furthermore, the model of the target vehicle, the vehicle color, and the like may be included.

In addition, the control unit acquires provided information including an image of the target vehicle specified based on the vehicle-related information from the cooperating vehicle. The cooperating vehicle is equipped with an on-board camera, and the license plate information is recognized by the image recognition function included in the image captured by the on-board camera. A known technique can be used to recognize the license plate information. The cooperating vehicle transmits to the information processing device provision information including an image of the target vehicle specified based on the vehicle-related information. The image of the target vehicle includes at least one of the following: passengers riding in the target vehicle, the color of the target vehicle, stickers attached to the body of the target vehicle, decorations provided on the target vehicle, and scratches on the body of the target vehicle. Contains images that are identifiable. Further, the provided information includes time information when the target vehicle is detected, position information, and road link information.

Then, the control unit transmits feature information about the feature of the target vehicle identified from the image to the cooperating vehicle. The cooperating vehicle can identify the target vehicle based on the feature information regarding the feature of the target vehicle transmitted from the information processing device. For example, even if it is difficult to recognize license plate information, such as a vehicle traveling in the opposite lane of the target vehicle, the target vehicle can be detected based on the feature information. Vehicles can be tracked even if the license plate is no longer detectable.

Embodiments of the present disclosure will be described below with reference to the drawings. The configurations of the following embodiments are examples, and the present disclosure is not limited to the configurations of the embodiments. Moreover, the following embodiments can be combined as much as possible.

<First Embodiment>
FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 according to this embodiment. The information processing system 1 is a system that enables tracking of a tracked vehicle based on a feature image of the tracked vehicle other than license plate information.

The information processing system 1 illustrated in FIG. 1 includes a center server 10 and in-vehicle devices 20 (#1 to #3) mounted on vehicles 2 (#1 to #3), respectively. The center server 10 and the in-vehicle devices 20 (#1 to #3) mounted on each vehicle are interconnected by a network N1. Hereinafter, the vehicles 2 (#1 to #3) are also collectively referred to as the vehicle 2, and the in-vehicle devices 20 (#1 to #3) are also collectively referred to as the in-vehicle devices 20. The center server 10 is an information processing device that manages a plurality of vehicles 2 on which the in-vehicle devices 20 are mounted. The in-vehicle device 20 is a control device capable of communicating with the center server 10 through the network N1 and capable of executing functions to be described later.

The network N1 is, for example, a worldwide public communication network such as the Internet, and may employ a WAN (Wide Area Network) or other communication networks. The network N1 may also include a telephone communication network such as a mobile phone, a wireless communication network such as Wi-Fi (registered trademark), and a dedicated communication network such as a VPN (Virtual Private Network). Although one center server 10 is illustrated in FIG. 1, a plurality of center servers 10 that control different areas may be connected. Also, although three vehicles 2 (#1 to #3) each having an in-vehicle device 20 mounted thereon are illustrated, there may be three or more of these vehicles 2 .

FIG. 2 is a diagram showing an example of the hardware configuration of the center server 10 and the in-vehicle device 20 that configure the information processing system 1 according to this embodiment. As shown in FIG. 2, the center server 10 is a computer including a processor 101, a main storage section 102, an auxiliary storage section 103, a communication section 104, and an input/output IF 105, which are interconnected by a connection bus. The main storage unit 102 and the auxiliary storage unit 103 are recording media readable by the center server 10 . The main storage unit 102 and the auxiliary storage unit 103 constitute memory of the center server 10 . A plurality of the above components may be provided, or some of the components may be omitted.

The processor 101 is a central processing unit that controls the center server 10 as a whole. The processor 101 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or the like. processor 101
is an example of a control unit. The processor 101, for example, develops a program stored in the auxiliary storage unit 103 in a work area of the main storage unit 102 so that it can be executed, and controls peripheral devices through execution of the program to perform a function that meets a predetermined purpose. I will provide a.

The main storage unit 102 stores programs executed by the processor 101, data processed by the processor, and the like. The main storage unit 102 includes flash memory, RAM (Random Access Memory), and ROM (Read Only Memory). The auxiliary storage unit 103 is a silicon disk including a nonvolatile semiconductor memory (flash memory, EPROM (Erasable Programmable ROM)), a solid state drive device, a hard disk drive (HDD, Hard Disk Drive) device, or the like. The auxiliary storage unit 103 stores various programs and various data in a recording medium in a readable and writable manner. The auxiliary storage unit 103 is also called an external storage device. The auxiliary storage unit 103 is used as a storage area that assists the main storage unit 102, and stores programs executed by the processor 101, data processed by the processor 101, and the like. The auxiliary storage unit 103 stores, for example, an operating system (OS), various programs, various tables, and the like. The OS includes a communication interface program that exchanges data with devices connected via the communication unit 104 . Note that the center server 10 may be a single computer, or may be a combination of a plurality of computers. Information stored in the auxiliary storage unit 103 may also be stored in the main storage unit 102 . Information stored in the main storage unit 102 may also be stored in the auxiliary storage unit 103 . A series of processes executed by the center server 10 are executed by the processor 101 according to a program as described above. However, at least part of the series of processes can also be executed by hardware such as a digital circuit.

The communication unit 104 is means for communicating with the in-vehicle device 20 via the network N1. The communication unit 104 can adopt an appropriate configuration according to the connection method with the network N1. The communication unit 104 is, for example, a LAN (Local Area Network) interface board or a wireless communication circuit for wireless communication. The communication unit 104 can adopt an appropriate configuration according to the connection method with the network N1. The input/output IF 105 is an interface for inputting/outputting data with devices connected to the center server 10 . Input devices such as a touch panel, a mouse, and a keyboard are connected to the input/output IF 105 . Through the input/output IF 105, an operation instruction or the like from an operator who operates an input device is received. Further, the input/output IF 105 is connected to, for example, display devices such as LCD (Liquid Crystal Display) and EL (Electroluminescence) panels, and output devices such as printers and speakers.

The in-vehicle device 20 is a computer that can be mounted in the vehicle 2 . As shown in FIG. 2, the in-vehicle device 20 includes a processor 201, a main storage unit 202, an auxiliary storage unit 203, a communication unit 204, an input IF 205, a position information sensor 206, and an in-vehicle camera 207, which are interconnected by a connection bus. have. A plurality of the above components may be provided, or some of the components may be omitted. The processor 201, the main storage unit 202, the auxiliary storage unit 203, and the communication unit 204 are the same as the processor 101, the main storage unit 102, the auxiliary storage unit 103, and the communication unit 104 of the center server 10, respectively, so the description thereof is omitted. Similarly to the center server 10, the in-vehicle device 20 may be a single computer, or may be a combination of multiple computers. A series of processes executed by the in-vehicle device 20 are executed by the processor 201 according to a program. However, at least part of the series of processes can also be executed by hardware such as a digital circuit.

The input IF 205 is an interface with an in-vehicle network (for example, CAN (Controller Area Network), etc.) in the vehicle 2 . Via the input IF 205, the speed and the like of the vehicle 2 are obtained from an ECU (Electronic Control Unit) or the like connected to the in-vehicle network. The position information sensor 206 acquires position information (for example, latitude and longitude) of the vehicle 2 on which the in-vehicle device 20 is mounted. The position information sensor 206 is, for example, a GPS (Global Positioning System) receiver, a wireless LAN communication unit, or the like. The in-vehicle camera 207 is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor).
) Photographing is performed using an imaging device such as an image sensor. The in-vehicle camera 207 may be a front camera, a back camera, a side camera, or the like, but is preferably installed at a position capable of capturing images of a preceding vehicle, an oncoming vehicle, or a following vehicle. An image obtained by shooting may be either a still image or a moving image with a predetermined frame rate (eg, 30 fps).

In the information processing system 1 according to the present embodiment, the center server 10 transmits vehicle-related information about a vehicle to be tracked (hereinafter also referred to as a tracked vehicle or target vehicle) to the vehicle 2 on which the in-vehicle device 20 is mounted. The vehicle-related information includes at least license plate information (vehicle registration number marked on the license plate) about the target vehicle. Also, the vehicle-related information may include target area information that indicates a range in which to search for the target vehicle. The processing load on each vehicle 2 can be reduced by limiting the range in which the target vehicle is searched. Furthermore, the model of the target vehicle, the vehicle color, and the like may be included.

The vehicle-related information about the target vehicle transmitted from the center server 10 is received by the in-vehicle device 20 of the vehicle 2 via the network N1. The in-vehicle device 20 acquires image data captured by the in-vehicle camera 207 and performs recognition processing of license plate information included in the image. A known technique can be used to recognize the license plate information. If the vehicle-related information includes the type and color of the target vehicle, the target vehicle is sorted using this information as filter information, and the image data after sorting is subjected to license plate information recognition processing. may be

When the in-vehicle device 20 detects the target vehicle, the in-vehicle device 20 detects the captured image of the target vehicle, the management number (tag number) of the target vehicle, the vehicle ID of the own vehicle (license plate information, etc.), the time information, the position information, the road on which the vehicle is traveling, to the center server 10, including the road link number for identifying the After detecting the target vehicle, the in-vehicle device 20 shifts to a state of continuously transmitting the provided information of the target vehicle to the center server 10 (hereinafter also referred to as "tracking mode"). The in-vehicle device 20 shifted to the tracking mode periodically transmits the provided information to the center server 10 while tracking the target vehicle is continued. Further, when a new feature of the target vehicle is detected from the image data captured by the on-vehicle camera 207, the in-vehicle apparatus 20 that has shifted to the tracking mode extracts the feature image and transmits it to the center server 10 as provided information. . Such features include, for example, passengers of the target vehicle, attachments such as stickers, ornaments and cup holders provided in the vehicle, decorations such as dolls, and scars on the vehicle body.

FIG. 3 is a diagram for explaining tracking of a target vehicle (tracked vehicle). In FIG. 3, the target vehicle V1 is the vehicle to be tracked, and is traveling in the direction of the white arrow on the one-lane road R1. Vehicles 2 # 1 and 2 # 2 are vehicles equipped with in-vehicle device 20 , and each vehicle receives vehicle-related information transmitted from center server 10 . Vehicle 2#2 is a vehicle that precedes target vehicle V1, and the fan-shaped area indicated by the dashed line represents the viewing angle of vehicle-mounted camera 207 (rear camera). Vehicle 2#1 is a vehicle that travels behind target vehicle V1, and the fan-shaped area indicated by the thin dashed line represents the viewing angle of vehicle-mounted camera 207 (front camera). The vehicle 2#1 traveling behind the target vehicle V1 can also be called "a vehicle that follows the target vehicle V1".

FIG. 4 is a diagram for explaining features of the target vehicle. FIG. 4(a) illustrates a schematic visual view of the target vehicle V1 viewed from the rear side, and FIG. 4(b) illustrates a schematic visual view of the target vehicle V1 viewed from the front side. It is

The in-vehicle device 20 of the vehicle 2#1 acquires, for example, the image data shown in FIG. A target vehicle V1 is detected. Then, the in-vehicle device 20 of the vehicle 2#1 shifts to the tracking mode, and, for example, the vehicle body color shown in the area of the dashed line frame V1b, the attached object such as the sticker shown in the area of the dashed line frame V1c, and the area of the dashed line frame V1d. is detected as a feature. The detected features are extracted as image data and transmitted to the center server 10 as provided information. For example, the provided information is transmitted to the center server 10 each time a new feature regarding the target vehicle V1 is detected. However, the provided information may be accumulated in the main storage unit 202 or the like for a predetermined period and periodically transmitted to the center server 10 .

Further, the in-vehicle device 20 of vehicle 2#2 acquires, for example, the image data shown in FIG. to detect the target vehicle V1. Then, the in-vehicle device 20 of the vehicle 2#2 shifts to the tracking mode, and for example, the vehicle body color shown in the area of the dashed line frame V1b, the decorations hung inside the vehicle as shown in the area of the dashed line frame V1e, Crews and the like shown in the areas of dashed-line frames V1f and V1g are detected as features. The detected features are extracted as image data and transmitted to the center server 10 as provided information. In the vehicle 2#2, the provided information is also transmitted to the center server 10 each time a new feature regarding the target vehicle V1 is detected. However, the provided information may be accumulated in the main storage unit 202 or the like for a predetermined period and periodically transmitted to the center server 10 .

Note that the in-vehicle device 20 that has detected the target vehicle V1 may acquire the vehicle speed of its own vehicle via the in-vehicle network connected to the input I/F 205, and include the vehicle speed in the provided information and transmit it. In addition, the in-vehicle device 20 that has shifted to the tracking mode detects the lighting of the turn signal lamps when the target vehicle V1 turns left or right, and the lighting of the tail lights when the target vehicle V1 stops, etc., as events indicating the behavior of the target vehicle V1. Send to server 10 . In this case as well, the in-vehicle device 20 can include information indicating the vehicle speed of the own vehicle, the operation direction of the turn signal lever, the operation position of the brake pedal, and the like acquired via the in-vehicle network in the provided information.

The center server 10 collects the characteristic data of the target vehicle V1 transmitted from each vehicle, and accumulates it in a database. Then, the center server 10 classifies the feature data accumulated in the database into categories indicating types of features. The new characteristics of the classified target vehicle V1 are sent to each vehicle together with the road link information on which the target vehicle V1 is presumed to exist.

Returning to FIG. 3, vehicle-related information, characteristic information, and road link information of the target vehicle V1 are transmitted from the center server 10 to the vehicle 2#3 traveling in the opposite lane of the one-lane road R1. be done. For example, it is assumed that the feature information includes feature data indicated by dashed-line frames V1e, V1f, and V1g in FIG. 4(b).

The in-vehicle device 20 of vehicle 2#3, which is traveling in the opposite lane to the lane in which the target vehicle V1 is traveling, detects the target vehicle V1 from the image data captured by the in-vehicle camera 207 at the fan-shaped viewing angle indicated by the dashed-dotted line. do. For example, the in-vehicle device 20 of the vehicle 2#3 can detect the target vehicle V1 using the body color and characteristic information of the target vehicle V1 even if the license plate information cannot be recognized. The in-vehicle device 20 detects the target vehicle V1 by, for example, recognizing the vehicle body color shown in the area of the dashed frame V1b in FIG. be possible. Similarly, the in-vehicle device 20 can detect the target vehicle V1 by recognizing the crew member (driver) indicated by the dashed frame V1f and the crew member (driver assistant) indicated by the dashed frame V1g. Become.

As described above, in the information processing system 1 according to the present embodiment, for example, even if it is difficult to recognize the license plate information, such as a vehicle traveling in the oncoming lane of the target vehicle V1, the target vehicle V1 is based on the feature information. Vehicle detection is possible. In addition, for example, from a vehicle traveling in the opposite lane of the target vehicle V1, characteristics that cannot be detected by the following vehicle like the vehicle 2#1 (characteristics of the driver and driver assistant, attached objects such as stickers, ornaments, scars, etc.) ) can be detected, allowing more features to be collected and accumulated. According to the information processing system 1 according to the present embodiment, it is possible to acquire highly accurate information on the target vehicle. Therefore, the information processing system 1 can accurately track the vehicle.

(Functional configuration)
Next, functional configurations of the center server 10 and the in-vehicle device 20 of the information processing system 1 according to the present embodiment will be described with reference to FIGS. 5 to 9. FIG.

(Center server)
FIG. 5 is a diagram schematically showing an example of the functional configuration of the center server 10. As shown in FIG. The center server 10 includes a vehicle-related information transmission unit 11, a provided information acquisition unit 12, a feature data processing unit 13, a target vehicle DB 14, a provided information DB 15, a feature information DB 16, and a map information DB 17 as functional components. The processor 101 of the center server 10 executes the processes of the vehicle-related information transmitting section 11, the provided information acquiring section 12, and the characteristic data processing section 13 by the computer program developed in the main storage section 102. FIG. However, any one of the functional constituents or part of the processing thereof may be executed by hardware such as a digital circuit.

The tracked vehicle DB 14, the provided information DB 15, and the characteristic information DB 16 are constructed by managing data stored in the auxiliary storage unit 103 by a database management system (DBMS) program executed by the processor 101. . The tracked vehicle DB 14, the provided information DB 15, and the feature information DB 16 are, for example, relational databases. The map information DB 17 stores map data including feature positions such as road link numbers and node numbers such as intersections, and map information including POI information such as characters and photographs indicating the characteristics of each point on the map data. be. The map information DB 17 may be provided from another system connected to the network N1, such as a GIS (Geographic Information System).
Also, any one of the functional components of the center server 10 or part of the processing thereof may be executed by another computer connected to the network N1.

The vehicle-related information transmission unit 11 refers to the tracked vehicle DB 14 and transmits vehicle-related information regarding the target vehicle to the vehicle 2 in which the in-vehicle device 20 is mounted. The tracked vehicle DB 14 stores vehicle-related information associated with an identification number (tag number) for the center server 10 to manage the target vehicle. The vehicle-related information includes at least license plate information about the target vehicle. Further, the vehicle-related information transmission unit 11 refers to the characteristic information DB 16 and transmits characteristic information regarding the target vehicle to the vehicle 2 . The characteristic information about the target vehicle is transmitted together with the vehicle-related information of the target vehicle and road link information on which the target vehicle is presumed to exist.

The provided information acquisition unit 12 acquires provided information transmitted from the vehicle 2 that has detected the target vehicle. The provided information includes the captured image of the target vehicle, the management number (tag number) of the target vehicle, the vehicle ID of the own vehicle (license plate information, etc.), the time information, the position information, the road link number that identifies the road on which it is traveling, etc. is included. The provided information also includes new feature information (feature data, image data indicating occurrence of events related to behavior such as turning left or right or stopping, vehicle speed, etc.) detected from the target vehicle. The provided information acquisition unit 12 stores the acquired provided information in the provided information DB 15 in association with the time information. The provided information of the target vehicle is collected and accumulated for each management number.

The feature data processing unit 13 refers to the provided information DB 15, aggregates the feature data and the imaging data collected from each vehicle for each target vehicle, and classifies them into categories indicating types of features. The classified features are stored in the feature information DB 16 . Further, the feature data processing unit 13 refers to the map information DB 17 and estimates the road link on which the target vehicle is located from the provided information indicating the occurrence of an event related to behavior such as turning left or right or stopping. Then, the feature data processing unit 13 identifies road links around the road link from the estimated road link. The identified road link is stored in the tracked vehicle DB 14 .

The tracked vehicle DB 14 stores vehicle-related information associated with the management number of the vehicle to be tracked. The vehicle-related information includes at least license plate information about the target vehicle. Also, the vehicle-related information may include target area information that indicates a range in which to search for the target vehicle. The processing load on each vehicle 2 can be reduced by limiting the range in which the target vehicle is searched. The target area information is, for example, regional mesh information such as a standard regional mesh code and a divided regional mesh code. However, the target area information may be information indicating a municipality or the like, or may be information separated by latitude and longitude for specifying an area. Further, when the vehicle type, vehicle color, etc. are specified in the initial stage of tracking, such information may be included in the vehicle-related information. Furthermore, the target vehicle is detected, and the road link information narrowed down and specified from the road links on which the presence of the target vehicle is estimated is added to the vehicle-related information.

FIG. 6 is a diagram showing an example of provided information stored in the provided information DB 15. As shown in FIG. The provided information is managed as a provided information table for each target vehicle, as illustrated in FIG. Information registered in the provided information table can be appropriately added, changed, or deleted. In FIG. 6, the provided information table has fields of acquisition date and time, cooperating vehicle ID, position information, link No., image information, and event. The image information field further has subfields in which a plurality of images are stored. The number of subfields in the image information field can also be appropriately added, changed, or deleted.

Acquisition date and time stores information indicating the acquisition date and time of the provided information transmitted from the in-vehicle device 20 that has detected the target vehicle. In FIG. 6, the date and time of acquisition are shown in units of minutes, but the date and time of acquisition may be in units of seconds, 5 minutes, or 30 minutes. The provided information table is updated with the latest acquisition date and time. The cooperating vehicle ID stores identification information (cooperating vehicle ID) for uniquely identifying the vehicle 2 managed by the center server 10 and capable of cooperating in tracking. The cooperating vehicle ID may be the automobile registration number of the vehicle 2 . The location information stores location information (latitude, longitude, etc.) indicating the location of the vehicle 2 that has transmitted the provided information. The link No stores information indicating the road link number on which the vehicle 2 that has transmitted the provided information exists. Note that the link No. may include a node number such as an intersection, an address, and the like. The image information stores information indicating the image data of the target vehicle imaged via the vehicle-mounted camera 207, which is included in the provided information. As the image information, for example, image data such as vehicle body color, attachments such as stickers, scars, decorations, crew members (driver, driving assistant), etc. are stored. The event stores image data indicating the behavior of the target vehicle (lighting of turn signal lamps when turning left or right, lighting of taillights when the vehicle stops, etc.). Images stored in image information and events may be still images or moving images. If the provided information includes information indicating the vehicle speed, the operation direction of the turn signal lever, the operation position of the brake pedal, etc., the information may be stored in association with the image information or the event. When there is no image information or information stored in the event, information such as "blank" or "---" indicating that the information does not exist is stored.

FIG. 7 is a diagram showing an example of feature information stored in the feature information DB 16. As shown in FIG. The feature information is managed as a feature information table as illustrated in FIG. Information registered in the feature information table can be appropriately added, changed, or deleted. In FIG. 7, the characteristic information table has fields of vehicle ID and characteristic information. The feature information field further has subfields in which images of each type representing features are stored. The number of subfields in the feature information field can also be added, changed, or deleted as appropriate.

The vehicle ID stores a management number that identifies the vehicle to be tracked. The feature information stores the feature data of the target vehicle aggregated and classified by the feature data processing unit 13 . The color subfield stores information indicating the aggregated vehicle color. Here, the aggregated vehicle color is, for example, the color determined by majority vote when the feature data of one tracked vehicle collected from each vehicle includes the color of a plurality of vehicles. For example, if the appearance of one tracked vehicle includes a plurality of colors, one or more vehicle colors may be determined in order from the color that occupies the largest area. In addition, in the provided information collected from a plurality of vehicles, when different vehicle colors are included for one vehicle to be tracked, one or more vehicle colors may be determined in descending order of the number of times provided. Similarly, for Passenger A, aggregated image data indicating the driver, for Passenger B, aggregated image data indicating the driver's assistant, and for the pasted matter, aggregated image data indicating a sticker or the like, decorations, etc. Objects store image data representing aggregated decorations in the vehicle. In addition, image data representing aggregated scars is stored in scars. Here, the aggregated image data refers to a set of image data from which image data having similar features are removed. Therefore, the image data stored in the feature information may be single or plural. Further, when there is no image data stored in the feature information, information such as "blank" or "---" indicating that the information does not exist is stored. When image data having similar features are removed, image data with clear contrast and the like may be preferentially left.

(In-vehicle device)
Next, the in-vehicle device 20 will be described. FIG. 8 is a diagram schematically showing an example of the functional configuration of the in-vehicle device 20. As shown in FIG. The in-vehicle device 20 includes a vehicle-related information acquisition unit 21, an image information acquisition unit 22, a tracking processing unit 23, an information provision processing unit 24, a target vehicle DB 25, a vehicle image DB 26, and a map information DB 27 as functional components. The processor 201 of the in-vehicle device 20 executes the processes of the vehicle-related information acquisition section 21, the image information acquisition section 22, the tracking processing section 23, and the information provision processing section 24 according to the computer program developed in the main storage section 202. FIG. However, any one of the functional constituents or part of the processing thereof may be executed by hardware such as a digital circuit.

The target vehicle DB 25 and the vehicle image DB 26 are constructed by managing data stored in the auxiliary storage unit 203 by a database management system program executed by the processor 201 . The target vehicle DB 25 and the vehicle image DB 26 are, for example, relational databases. The map information DB 27 is, for example, a database provided by a navigation system connected via an in-vehicle network. The map information DB 27 stores map data including feature positions such as road link numbers and node numbers such as intersections, and map information including POI information such as characters and photographs indicating the characteristics of each point on the map data. be.

The vehicle-related information acquisition unit 21 acquires vehicle-related information regarding the target vehicle transmitted from the center server 10 . The acquired vehicle-related information is stored in the target vehicle DB 25 . Further, the vehicle-related information acquisition unit 21 acquires feature information regarding the target vehicle transmitted from the center server 10 . The acquired feature information is stored in the target vehicle DB 25 in association with the management number of the target vehicle.

The image information acquisition unit 22 acquires a camera image captured by the vehicle-mounted camera 207 when the position of the own vehicle satisfies a predetermined condition. For example, the image information acquisition unit 22 refers to the target vehicle DB 25 and the map information DB 27, and the position information (latitude, longitude) of the own vehicle acquired from the position information sensor 206 is included in the target area specified by the vehicle-related information. Acquire the camera image if possible. Similarly, the image information acquisition unit 22 acquires a camera image when the position information of the own vehicle acquired from the position information sensor 206 corresponds to the road link information designated by the vehicle-related information. The image information acquiring unit 22 stores the acquired camera image in the vehicle image DB 26 in association with the time information at which the image was acquired, the position information, the road link information, and the management number of the target vehicle.

The tracking processing unit 23 refers to the target vehicle DB 25 and the vehicle image DB 26 to detect vehicles to be tracked. The tracking processing unit 23 acquires image data stored in the vehicle image DB 26, for example, and performs recognition processing of license plate information, feature information, vehicle color, and vehicle type included in the image. A known technique can be adopted for these image recognition processes. When the target vehicle is detected, the tracking processing unit 23 transfers the management number and image data of the detected target vehicle to the information provision processing unit 24, and continuously transmits the provided information of the target vehicle to the center server 10. (tracking mode). While the target vehicle is being detected, the tracking processing unit 23 that has shifted to the tracking mode periodically transfers the management number and image data of the target vehicle to the information providing processing unit 24 .

The tracking processing unit 23 shifted to the tracking mode detects new characteristics of the target vehicle from the camera image captured by the vehicle-mounted camera 207 . For example, when the tracking processing unit 23 detects a feature such as a passenger of the target vehicle, an attached object such as a sticker, an ornament provided in the vehicle, a cup holder, an ornament such as a doll, or a scar on the vehicle body, the tracking processing unit 23 detects the feature. is extracted and handed over to the information provision processing unit 24 .

Further, when the tracking processing unit 23 that has shifted to the tracking mode detects an event indicating the behavior of the target vehicle such as turning left or right or stopping from the camera image captured by the vehicle-mounted camera 207, image data indicating the event is detected. is extracted and handed over to the information provision processing unit 24 . The tracking processing unit 23 detects, for example, the turn-on of the turn signal lamp when the target vehicle turns left or right, and the turn-on of the tail light when the target vehicle stops, as an event indicating the behavior.

The information provision processing unit 24 adds the image data handed over from the tracking processing unit 23, the management number of the target vehicle, the time information when the image was acquired, the position information, the road link information, and the vehicle ID of the own vehicle (cooperating vehicle ID). to the center server 10. The information provision processing unit 24 may include information indicating the vehicle speed, the vehicle speed of the own vehicle, the operation direction of the turn signal lever, the operation position of the brake pedal, and the like acquired through the in-vehicle network in the provided information and transmit the information. .

FIG. 9 is a diagram showing an example of information stored in the target vehicle DB 25. As shown in FIG. As illustrated in FIG. 9, the information stored in the target vehicle DB 25 is managed as a vehicle information table in which lines are separated for each target vehicle management number. Information registered in the vehicle information table can be appropriately added, changed, or deleted. In FIG. 9, the vehicle information table has fields for No, vehicle ID, license plate information, target area, and feature information. The feature information field has subfields in which an image for each type indicating features is stored. The number of subfields in the feature information field can also be added, changed, or deleted as appropriate.

In No, a number assigned by the in-vehicle device 20 for managing the target vehicle is stored. The vehicle ID stores a management number that identifies the vehicle to be tracked by the center server 10 . The license plate information stores information indicating the vehicle registration number marked on the license plate of the target vehicle. The target area stores target area information that indicates a range in which to search for a target vehicle. The target area information is, for example, regional mesh information such as a standard regional mesh code and a divided regional mesh code. However, the target area information may be information indicating a municipality or the like, or may be information separated by latitude and longitude for specifying an area.

The feature information stores the feature data of the target vehicle aggregated and classified by the center server 10 and the specified road link information. The color subfield stores information indicating the aggregated vehicle color. Similarly, for Passenger A, aggregated image data indicating the driver, for Passenger B, aggregated image data indicating the driver's assistant, and for the pasted matter, aggregated image data indicating a sticker or the like, decorations, etc. Objects store image data representing aggregated decorations in the vehicle. In addition, image data representing aggregated scars is stored in scars. The link No. subfield stores road link information specified by narrowing down road links where the presence of the target vehicle is estimated. The image data and road links stored in the feature information may be single or plural. Further, when there is no data stored in the feature information, information such as "blank" or "---" indicating that the information does not exist is stored.

(Processing flow)
Processing of the information processing system 1 according to the present embodiment will be described with reference to FIGS. 10 and 11 are sequence diagrams showing an example of the flow of processing executed by the center server 10 and the in-vehicle device 20. FIG.

In FIG. 10, the center server 10 simultaneously transmits vehicle-related information about vehicles to be tracked to the vehicles 2 on which the in-vehicle devices 20 are mounted (S1). The vehicle-related information includes at least a tag number (vehicle ID in FIG. 9) and license plate information for the center server 10 to manage the target vehicle. In addition, when there is information such as vehicle color, vehicle type, and target area information indicating the range to search for the target vehicle, such information is included in the vehicle-related information and transmitted. The vehicle-related information about the target vehicle transmitted from the center server 10 is received by the vehicle 2 on which the in-vehicle device 20 is mounted (S2).

The in-vehicle device 20 stores the received vehicle-related information in the target vehicle DB 25 . The in-vehicle device 20 refers to the target vehicle DB 25 and acquires features of the target vehicle such as license plate information, vehicle type, and vehicle color. Then, the in-vehicle device 20 acquires the camera image captured by the in-vehicle camera 207, and collates the vehicle that matches the license plate information and characteristics of the target vehicle by image recognition (S3). When the target vehicle is detected as a result of the collation, the in-vehicle device 20 shifts to a state (tracking mode) in which information on the target vehicle is continuously transmitted to the center server 10, and shifts to the processing of S5 (YES in S4). . Then, the in-vehicle device 20 adds at least the camera image of the detected target vehicle to the tag number of the target vehicle, identification information of the own vehicle ( FIG. 6 , cooperating vehicle ID), time information, position information (latitude, longitude), road Provided information associated with the link information is transmitted to the center server 10 . If the target vehicle is not detected, the in-vehicle device 20 returns the process to S3 (NO in S4), and continues to collate vehicles that match the license plate information and characteristics of the target vehicle.

The in-vehicle device 20 that has shifted to the tracking mode periodically provides the center server 10 with provision information in which the camera image of the target vehicle is associated with the tag number of the target vehicle, the identification information of the own vehicle, the time information, the position information, and the road link information. (S5). In addition, when new features (passengers of the target vehicle, stickers and other attachments, decorations and cup holders installed in the vehicle, decorations such as dolls, scars on the vehicle body, etc.) are detected, images containing such features Data is extracted and transmitted to the center server 10 . The image data including the new features is transmitted as provided information together with the tag number of the target vehicle, the identification information of the own vehicle, the time information, the position information, and the road link information. When a new feature is detected, the in-vehicle device 20 that continues the tracking process extracts image data including the new feature, and extracts the tag number of the target vehicle, the identification information of the own vehicle, the time information, the location, and the like. Information and road link information are sent to the center server 10 (S6). The provided information may include information such as the vehicle speed of the own vehicle.

The provided information transmitted from the in-vehicle device 20 is received by the center server 10 (S7). The center server 10 stores the provided information DB 15 transmitted from the in-vehicle device 20 . In the provided information DB 15, the provided information transmitted from the in-vehicle device 20 is sorted in order of acquisition date and time, collected and accumulated, and managed as a provided information table for each target vehicle.

The center server 10 aggregates the feature data and imaging data collected from each vehicle and stored in the provided information DB 15 for each target vehicle, and classifies them into categories indicating types of features. The classified features are stored in the feature information DB 16 (S8). Then, the center server 10 transmits the feature data indicating the added feature and the road link information on which the target vehicle exists to each vehicle (S9). Feature data indicating added features and road link information on which the target vehicle exists, which are transmitted from the center server 10, are received by the vehicle-mounted device 20, and the target vehicle DB 25 managed by the vehicle-mounted device is stored as the latest information. .

Note that the center server 10 estimates the current location of the target vehicle from the location information and road link information included in the provided information, and transmits characteristic data to vehicles within a predetermined area including the location. good too. A predetermined area is designated as target area information. However, as will be described later with reference to FIG. 12, the target area information may specify the center position (latitude and longitude) of a predetermined area and the distance radius (for example, 10 km). By specifying the target area and narrowing down the range in which the target vehicle exists, it is possible to improve the tracking accuracy. Further, the center server 10 may transmit the characteristic information to vehicles on the road link around the location of the target vehicle, and instruct the continuation of the tracking process only to the vehicles located on the road link. may be sent. Also in this case, the tracking accuracy of the target vehicle can be improved.

When the vehicle 2 for which the tracking process continues detects an event indicating the behavior of the target vehicle (turn left or right, stop), it extracts image data indicating the occurrence of the event and transmits it to the center server 10 as provided information ( S10). The provided information includes the tag number of the target vehicle, identification information of the own vehicle, time information, position information, and road link information. The vehicle 2 for which the tracking process continues includes the vehicle-mounted device 20 that has shifted to the tracking mode after receiving the information specifying the road link information together with the added feature data. In the case of the vehicle 2 that turns left and right following the target vehicle, the provided information includes information indicating the node number, the vehicle speed acquired via the in-vehicle network, the operation direction of the turn signal lever, the operation position of the brake pedal, and the like. may be included.

The center server 10 receives the provided information accompanying the occurrence of the event (S11). The provided information transmitted from the in-vehicle device 20 with the occurrence of the event is stored in the provided information DB 15 . Then, the center server 10 refers to the map information DB 17 and estimates a road link on which the target vehicle can exist based on the detected behavior of the target vehicle and road links and node information included in the provided information (S12).

The center server 10 further refers to the map information DB 17 to identify road links around the location where the target vehicle is estimated to exist. Then, the feature information about the target vehicle and an instruction to perform the tracking process are transmitted only to the vehicles on the specified specific road link (S13). The specific road link information transmitted from the center server 10 is received by the in-vehicle device 20 .

Moving on to FIG. 11, the center server 10 determines whether or not to end tracking (S14 in FIG. 11). If the center server 10 does not end the tracking, it returns to the process S7 and waits for reception of the next provided information. On the other hand, when terminating the tracking, the center server 10 simultaneously transmits to the vehicles 2 an instruction to terminate the tracking of the vehicle to be tracked (S15). Then, the center server 10 ends tracking. If the vehicle 2 does not receive the instruction to end tracking (NO in S16), the process returns to S3. Then, the in-vehicle device 20 of the vehicle 2 traveling on the specific road link detects the target vehicle (S3), and the target vehicle is tracked (S5-S6, S10). On the other hand, when the vehicle 2 receives an instruction to end tracking (YES in S16), the process ends.

Through the above processing, the information processing system 1 according to the present embodiment can perform the following operations: can be acquired by each vehicle (cooperating vehicle) and transmitted to the center server 10 . The center server 10 can image-analyze and summarize the features of the target vehicle collected and accumulated from each vehicle, and can transmit the aggregated feature data to vehicles around the target vehicle. When a vehicle that matches the characteristics of the target vehicle is detected, the imaging data of the vehicle can be uploaded to the center server 10, so tracking of the target vehicle can be made more efficient.

In the information processing system 1 according to the present embodiment, even if it is difficult to recognize the license plate information, such as a vehicle traveling in the opposite lane of the target vehicle, the target vehicle can be detected from the characteristic data of the vehicle. become. In addition, even if the information of the vehicle to be tracked is only the license plate information, the vehicle position and driving road can be grasped from the vehicle that detected the target vehicle by the license plate information, and the area around the target vehicle and the road link can be narrowed down. Since the information processing system 1 can accumulate the characteristics of the target vehicle detected at each viewing angle from vehicles traveling in the narrowed-down regional area and road link, it is possible to obtain information on the target vehicle with higher accuracy. be possible.

In this embodiment, the provided information transmitted from the in-vehicle device 20 to the center server 10 includes the tag number of the target vehicle, identification information of the cooperating vehicle (cooperating vehicle ID), time information, position information (latitude, longitude), May include road link information. Therefore, in the center server 10, the location of the target vehicle can be estimated in chronological order using the acquired tag number of the target vehicle, identification information of cooperating vehicles, time information, position information, and road link information. .

In this embodiment, the center server 10 can transmit the feature information to the vehicle 2 within a predetermined area including the location of the target vehicle estimated from the provided information. By narrowing down the target area to which the feature information is transmitted to a predetermined area including the location of the target vehicle, the processing load related to vehicle tracking can be reduced and the detection accuracy of the target vehicle can be increased. The center server 10 may further transmit the characteristic information to the vehicles 2 positioned on the road link around the location of the target vehicle identified from the provided information. It is expected that the processing load related to vehicle tracking will be further reduced, and the detection accuracy of the target vehicle will be improved.

(Modification)
The center server 10 may display the detected position of the target vehicle transmitted from each vehicle on a display device such as an LCD connected via the input/output I/F 105 . FIG. 12 is a diagram illustrating an example of the target vehicle displayed on the display device. In FIG. 12, the black circles represent the positions at which the target vehicle was detected, and t1, t2, and t3 numbered to the black circles represent the times at which the target vehicle was detected.

As shown in FIG. 12, by plotting the travel route of the target vehicle on a map at regular intervals, the trajectory of the target vehicle and the information on the area around the travel route can be tracked while visually recognizing it in time series. can. The center server 10 may plot on a map using the time information, position information, and road link information of the provided information accumulated in the provided information DB 15 for each management number of the target vehicle.

For example, if the target vehicle detected at time t1 on road link R2 is displayed at time t2, the person in charge of tracking knows that the target vehicle is moving along road link R2 in the direction of node N1. I can grasp it. Also, the person in charge can grasp the aspect of the surrounding area such that the road link R2 where the target vehicle is detected intersects with the road link R3 at the node N1. For example, the person in charge designates a predetermined range (t2) surrounded by a dashed line and instructs the center server 10 to transmit the vehicle-related information and feature information of the target vehicle. In this way, the tracking accuracy can be improved by notifying the target vehicle information in advance to the vehicles traveling on the road links R2 and R3 around the node N1 where the target vehicle is expected to travel. Note that the predetermined range (t2) is an example of designating a distance radius centered on the node N1, for example.

Next, when the target vehicle detected on road link R2 is displayed at the position at time t3, it can be understood that the target vehicle is moving on road link R2 in the direction of node N2. The road link R2 on which the target vehicle is detected intersects with the road link R4 and the road link R5 at the node N2. For example, the person in charge designates a predetermined range (t3) surrounded by a dashed line and instructs the center server 10 to transmit the vehicle-related information and characteristic information of the target vehicle. Alternatively, the center server 10 may be instructed to transmit the vehicle-related information and characteristic information of the target vehicle by designating the road links R2, R4, and R5. In this case as well, the information of the target vehicle can be notified in advance to the vehicles traveling on the road links R2, R4, and R5 around the node N2 where the target vehicle is expected to travel.

<Other embodiments>
The above-described embodiment is merely an example, and the present disclosure can be modified as appropriate without departing from the scope of the present disclosure. The processes and means described in the present disclosure can be freely combined and implemented as long as there is no technical contradiction.

Also, the processing described as being performed by one device may be shared and performed by a plurality of devices. Alternatively, processes described as being performed by different devices may be performed by one device. In a computer system, it is possible to flexibly change the hardware configuration (server configuration) to implement each function. For example, the center server 10 may have a part of the functions of the in-vehicle device 20 (for example, the image recognition function).

The present disclosure can also be implemented by supplying a computer program implementing the functions described in the above embodiments to a computer, and reading and executing the program by one or more processors of the computer. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. A non-transitory computer readable storage medium is any type of disk such as, for example, a magnetic disk (floppy disk, hard disk drive (HDD), etc.), an optical disk (CD-ROM, DVD disk, Blu-ray disk, etc.), It includes any type of medium suitable for storing electronic instructions, such as read only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic card, flash memory, or optical card.

1 information processing system 2 vehicle (cooperating vehicle)
10 center server (information processing device)
11 vehicle-related information transmitting unit 12 provided information acquiring unit 13 feature data processing unit 14 tracked vehicle database 15 provided information database 16 feature information database 20 in-vehicle device (control device)
21 vehicle-related information acquisition unit 22 image information acquisition unit 23 tracking processing unit 24 information provision processing unit 25 target vehicle database 26 vehicle image database

Claims (1)

transmitting vehicle-related information about the target vehicle being tracked to the cooperating vehicle;
Acquiring provided information including an image of the target vehicle identified based on the vehicle-related information from the cooperating vehicle;
transmitting feature information about the features of the target vehicle identified from the image to the cooperating vehicle;
An information processing apparatus comprising a control unit that executes

Images