JP2023139701A - Imaging device, imaging method, and program - Google Patents

Abstract

To provide a technology making it possible to store necessary images in response to an intention of a person who collects images from plural vehicles.SOLUTION: An imaging device mounted in a vehicle includes an imaging unit that images the outside of the vehicle and stores taken image data in a memory unit, a receiving unit that receives a lock instruction, which instructs a requirement for image data whose deletion is prohibited, from an information processor that acquires image data items from plural vehicles including the vehicle, and an image management unit that, when the lock instruction is received, brings image data, which meets the requirement, out of the image data items stored in the memory unit to a condition where deletion is prohibited.SELECTED DRAWING: Figure 6

Description

The present invention relates to a photographing device, a photographing method, and a program.

BACKGROUND ART Currently, an in-vehicle device called a drive recorder is provided that can record images of the direction of travel and the like while the vehicle is running (for example, Patent Document 1).

Japanese Patent Application Publication No. 2014-229049

Currently, many vehicles are traveling on the roads, and the drive recorders installed in these vehicles capture images of various places. Therefore, it is considered possible to obtain information about various locations by collecting and analyzing images from a large number of these vehicles.

For example, if there is a location that is likely to be monitored intensively, images taken by vehicles passing through the location may be collected at a later date, if necessary. However, since there is a limit to the storage capacity of a drive recorder for a certain period of time, there is a problem in that images recorded by a drive recorder are deleted after a certain period of time.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a technique that allows necessary images to be saved according to the intention of a person who collects images from a plurality of vehicles.

A photographing device according to one aspect of the present invention is a photographing device mounted on a vehicle, and includes a photographing section that photographs the outside of the vehicle and stores the photographed image data in a storage section, and a plurality of photographic devices including the vehicle. a receiving unit that receives a locking instruction that instructs conditions for image data that is prohibited from being deleted from an information processing device that collects image data from a vehicle; and image data that is stored in the storage unit when the locking instruction is received. The image management unit changes the image data that satisfies the conditions to a state in which deletion is prohibited.

According to the present invention, it is possible to provide a technique that allows necessary images to be saved according to the intention of a person who collects images from a plurality of vehicles.

FIG. 1 is a diagram showing an example of an image processing system according to the present embodiment. FIG. 2 is a diagram showing an example of the hardware configuration of a photographing device. 1 is a diagram illustrating an example of a hardware configuration of an image processing device. It is a figure showing an example of functional block composition of an imaging device. It is a diagram showing an example of a collected data DB and an instruction content DB. FIG. 2 is a diagram showing an example of a functional block configuration of an image processing device. It is a diagram showing an example of a collected data DB and a detection result DB. 3 is a flowchart showing a processing procedure performed by the image processing device in the image collection/lock function (part 1). FIG. 3 is a diagram illustrating an example of a vehicle search screen that accepts an instruction to upload or lock image data. FIG. 3 is a diagram illustrating an example of a vehicle search screen that accepts an instruction to upload or lock image data. FIG. 3 is a diagram for explaining a shooting range. FIG. 3 is a diagram for explaining overlapping conditions. FIG. 3 is a diagram for explaining overlapping conditions. FIG. 6 is a diagram for explaining a method of calculating a time period during which a location to be uploaded or locked was photographed. 2 is a flowchart showing the processing procedure performed by the photographing device in the image collection/lock function (part 1). 12 is a flowchart showing a processing procedure performed by the image processing device regarding the image collection/lock function (part 2). 12 is a flowchart showing the processing procedure performed by the photographing device regarding the image collection/lock function (part 2). 12 is a flowchart showing the processing procedure performed by the photographing device regarding the image collection/lock function (part 2). 2 is a flowchart illustrating an example of a processing procedure when the image processing device displays image data. It is a figure showing an image display screen (part 1). FIG. 3 is a diagram for explaining an image data extraction method. FIG. 3 is a diagram for explaining an image data extraction method. It is a figure showing an image display screen (part 2). 2 is a flowchart illustrating an example of a processing procedure when an image processing device performs image detection. FIG. 3 is a diagram illustrating an example of a screen that displays detected image detection targets.

Embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in each figure, those with the same reference numerals have the same or similar configurations.

<System configuration>
FIG. 1 is a diagram showing an example of an image processing system 1 according to the present embodiment. The image processing system 1 includes a photographing device 10, an image processing device 20, and a terminal 30. The photographing device 10 and the image processing device 20 can communicate with each other using, for example, wireless communication. Further, the image processing device 20 and the terminal 30 can communicate with each other by using, for example, wired communication or wireless communication.

Vehicles C include, but are not limited to, passenger cars, freight cars, trucks, buses, motorcycles, and the like. Any vehicle may be used as long as it can mount the photographing device 10 thereon.

The photographing device 10 grasps the position and direction in which the vehicle C is traveling, and periodically notifies the image processing device 20 of the position and direction in which the vehicle C is traveling. The position where the vehicle C is traveling can be measured using, for example, the photographing device 10 or a GNSS (Global Navigation Satellite System) or a GPS (Global Positioning System) mounted on the vehicle C. Further, the traveling direction of the vehicle C is determined using, for example, temporal changes in the position where the vehicle C is traveling, information from the photographing device 10 or various sensors (gyro sensor, geomagnetic sensor, etc.) mounted on the vehicle C. It can be measured by The photographing device 10 includes, but is not limited to, a drive recorder, an electronic mirror with a recording function, a mobile phone, a smartphone, a tablet terminal, etc., for example. For example, the photographing device 10 may be a part of the vehicle C. The photographing device 10 may be any device as long as it has a recording function.

Further, the photographing device 10 photographs the outside (surroundings) of the vehicle C, and stores the photographed image data in a memory held by the photographing device 10. The photographing device 10 is assumed to photograph a predetermined angle including the traveling direction of the vehicle C, like a front camera of a drive recorder, but is not limited thereto. For example, it may include photographing in the opposite direction to the traveling direction (that is, behind the vehicle C) like a rear camera of a drive recorder, or photographing in all directions including the traveling direction like a 360 degree camera. When the image capturing apparatus 10 runs out of memory space, it overwrites the oldest image data with new image data.

The photographing device 10 may continuously photograph still images at a predetermined period, or may photograph videos (moving images) at a predetermined frame rate. That is, the term "image" in this embodiment means a still image and/or video that is continuous in chronological order. Moreover, a "frame" means each still image which continues in chronological order, and each still image which comprises a video.

The image processing device 20 grasps the position where the vehicle C is traveling based on the notification from the photographing device 10, and collects image data from the plurality of vehicles C (from the photographing device 10 mounted on the plurality of vehicles C). . The image processing device 20 also performs analysis of the collected image data. Further, the image processing device 20 causes the terminal 30 to display image data collected from the photographing device 10.

In this embodiment, the image processing system 1 may include at least one of the following functions.
1. "Image collection/lock function": Image data captured by the image processing device 20 at a time and place designated by the user from the photographing device 10 of the vehicle C that was traveling in the time and place specified by the user. A function to collect (upload). In addition, the image processing device 20 locks the image data of the vehicle C that was traveling in the time and place specified by the user without deleting the image data taken at the place during the specified time. A function that instructs you to do something.
2. “Image display function”: A function for displaying image data collected by the image processing device 20 using the “image collection function” on the screen of the terminal 30. The image processing device 20 extracts image data taken at a user-specified location at a user-specified time, and displays it on the screen of the terminal 30 .
3. “Image detection function”: A function in which the image processing device 20 analyzes the image data collected from the photographing device 10 to detect a detection target (for example, a specific face, a specific license plate, etc.) in the image data. . Also, a function to search for a specified BLE tag among BLE (Bluetooth L Energy) tags detected by the photographing device 10.

In the following description, the location where the image data to be uploaded or locked is photographed is referred to as the "uploaded or locked location." In addition, the time period in which image data to be uploaded or locked is photographed is referred to as the "upload or lock target time period." When uploading and locking are explicitly used, they may be referred to as "upload target location," "upload target time zone," "lock target location," or "lock target time zone." Furthermore, in the following description, for convenience, the term "image data photographed by vehicle C" may be used, but more accurately, it means image data photographed by photographing device 10 mounted on vehicle C. Furthermore, for convenience, the term "image data collected from the vehicle C" may be used, but more accurately, it means the image data collected from the photographing device 10 mounted on the vehicle C. Further, for convenience, it may be written as "transmitting an instruction to the vehicle C," but more precisely, it means transmitting an instruction to the photographing device 10 mounted on the vehicle C.

<Hardware configuration>
(Photography equipment)
FIG. 2 is a diagram showing an example of the hardware configuration of the photographing device 10. As shown in FIG. The imaging device 10 includes a processor 11 such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), a memory, a storage device 12 such as an HDD (Hard Disk Drive) and/or an SSD (Solid State Drive), and wired or wireless communication. A communication IF (Interface) 13 that performs the above operations, a sensor 14 such as an acceleration sensor, a geomagnetic sensor, and/or a gyro sensor, an input device 15 that receives input operations, an output device 16 that outputs information, and a still image or video image. It has a camera 17. The input device 15 is, for example, an operation button, a touch panel, and/or a microphone. Further, the input device 15 may include an interface for inputting information measured by the vehicle C (vehicle speed, acceleration, etc.). The output device 16 is, for example, a display, a touch panel, and/or a speaker.

(Image processing device)
FIG. 3 is a diagram showing an example of the hardware configuration of the image processing device 20. As shown in FIG. The image processing device 20 includes a processor 21 such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), a memory, a storage device 22 such as an HDD (Hard Disk Drive) and/or an SSD (Solid State Drive), and a wired or wireless It has a communication IF (Interface) 23 for communicating, an input device 24 for accepting input operations, and an output device 25 for outputting information. The input device 24 is, for example, a keyboard, touch panel, mouse, and/or microphone. The output device 25 is, for example, a display, a touch panel, and/or a speaker.

The image processing device 20 may be configured using one or more physical servers, etc., may be configured using a virtual server running on a hypervisor, or may be configured using a cloud server. It may be configured using The image processing device 20 may also be called an information processing device.

<Functional block configuration>
(Photography equipment)
FIG. 4 is a diagram showing an example of the functional block configuration of the imaging device 10. The imaging device 10 includes a storage section 100, a measurement section 101, a notification section 102, an imaging section 103, a reception section 104, an image management section 105, a transmission section 106, and a detection section 107. The storage unit 100 can be realized using the storage device 12 included in the photographing device 10. Furthermore, the measurement unit 101 can be realized using the sensor 14 included in the imaging device 10. Further, the photographing unit 103 can be realized using the camera 17 included in the photographing device 10. In addition, the measuring section 101, the notification section 102, the photographing section 103, the receiving section 104, the image managing section 105, the transmitting section 106, and the detecting section 107 are the processor 11 of the photographing device 10, the storage device 12 This can be realized by executing a program stored in . Further, the program can be stored in a storage medium. The storage medium storing the program may be a non-transitory computer readable medium. The non-temporary storage medium is not particularly limited, but may be a storage medium such as a USB memory or a CD-ROM.

The storage unit 100 stores a collected data DB (DataBase) 100a that stores driving history and image data, and an instruction content DB 100b that stores instructions received from the image processing device 20.

FIG. 5 is a diagram showing an example of the collected data DB 100a and the instruction content DB 100b. The collected data DB 100a stores a driving history DB 100-1a and an image data DB 100-2a. The "traveling history" of the traveling history DB 100-1a is a history indicating when, where, and in which direction the vehicle C was traveling; for example, the traveling position, traveling direction, and time of the vehicle C. Data associated with each other is accumulated at predetermined intervals (for example, every 1 second, every 10 seconds, etc.). For example, the traveling position may be expressed by latitude and longitude, and the traveling direction may be expressed by a clockwise angle when a reference direction (for example, north direction) is set to 0 degrees. In the example of FIG. 5, data of "northern latitude x1, east longitude y1, z1 degree, 2021/10/22_10:10_00" is recorded as the travel history. This means that the traveling position at the photographing time of 10:10:00 on October 22, 2021 is x1 degree north latitude and y1 degree east longitude, and the traveling direction is z1 degree. In addition, the data for "north latitude x2, east longitude y2, z2 degrees, 2021/10/22_10:10_01" indicates that the driving position at the time of shooting at 10:10:01 on October 22, 2021 is north latitude x2 degrees and east longitude. This means that the direction of travel is 2 degrees y, and the direction of travel is 1 degree z. That is, in the example of FIG. 5, the driving history is recorded every second.

Further, "image data" of the image data DB 100-2a stores image data photographed by the photographing device 10. Further, the image data may include information indicating the shooting date and time of each frame. The lock flag indicates a time period in which image data is prohibited from being deleted. In the example of FIG. 5, the time period from 10:10:00 to 10:15:59 on October 22, 2021 is set to be prohibited from being deleted. Note that the method of recording the driving history and image data in the collected data DB 100a is merely an example, and the present embodiment is not limited to the example of FIG. 5. For example, all captured image data may be stored in one file, or may be divided into multiple files (for example, files for each minute) and stored. Further, in the example of FIG. 5, the driving history and image data are stored in separate databases, but the invention is not limited to this. A configuration may be adopted in which all data included in the driving history and image data are stored in one database.

The instruction content DB 100b stores an “instruction ID (Identifier)” that is an identifier for uniquely identifying an instruction from the image processing device 20. The "instruction ID" may be notified from the image processing device 20, or may be generated by the photographing device 10 itself. The “instruction content” field stores the content of the instruction from the image processing device 20. More specifically, whether to lock the image data or to upload the image data is stored. “Target location” stores information specifying at which location image data taken is to be locked or uploaded. In this embodiment, the method of specifying where the image data was taken may include a method of specifying a range (area) and a method of specifying a point. For example, "range specification" is an instruction to lock or upload image data taken from a specified range, and "point specification" is an instruction to lock or upload image data taken from a specified point. It may also be an instruction. “Target time period” stores information specifying when captured image data is to be locked or uploaded. Note that the designation of either the target location or the target time period may be omitted. Returning to FIG. 4, the explanation will be continued.

The measurement unit 101 measures the running position and running direction of the vehicle C on which the photographing device 10 is mounted. Furthermore, the measuring unit 101 associates the measured traveling position, traveling direction, and time of measurement of the vehicle C and stores them in the "travel history" of the collected data DB 100a.

The notification unit notifies the image processing device 20 of the running position, running direction, and measurement time of the vehicle C at predetermined intervals.

The photographing unit 103 photographs the outside of the vehicle and stores the photographed image data in the collected data DB 100a of the storage unit 100. Further, the photographing unit 103 may store photographed image data and photographing time in association with each other in the collected data DB 100a. For example, the photographing time at which each frame was photographed may be embedded in the image data stored in the collected data DB 100a. Further, the photographing unit 103 may associate the photographed image data with the traveling position and the traveling direction and store them in the collected data DB 100a.

The receiving unit 104 receives a lock instruction from the image processing device 20 that specifies conditions for image data that are prohibited from being deleted. The receiving unit 104 also receives an upload instruction to upload image data from the image processing device 20 .

When the image management unit 105 receives a lock instruction from the image processing device 20, the image management unit 105 prohibits deletion of image data that satisfies the conditions for image data that is prohibited from being deleted, among the image data stored in the collected data DB 100a. Change to state.

In addition, the lock instruction includes designation of a lock target time period as a condition for image data that is prohibited from being deleted, and the image management unit 105 selects the image data that was captured as image data that satisfies the conditions for image data that is prohibited from being deleted. Image data whose time corresponds to a designated target time period may be changed to a state where deletion is prohibited.

In addition, the lock instruction includes designation of a location to be locked as a condition for the image data to be prohibited from being deleted, and the image management unit 105 selects the lock target location from among the image data stored in the collected data DB 100a. Image data in which the photographing range generated from the photographing range based on the traveling position and traveling direction of the vehicle C and the target location satisfy the overlapping condition may be changed to a state in which deletion is prohibited. Moreover, the designation of the lock target location may be designation of a predetermined range (for example, 1 km square around a station, etc.). In this case, the image management unit 105 may change the image data in which the shooting range overlaps with at least a part of the predetermined range from being deleted as image data that satisfies the overlap condition. .

Further, the designation of the location to be locked may be designation of a predetermined point (for example, a specific building). In this case, the image management unit 105 may change the image data in which the predetermined point is included in the photographing range to a state in which deletion is prohibited as the image data that satisfies the duplication condition.

Further, the lock instruction may specify a predetermined point as a lock target location, and may also include a shooting direction specification as a condition for image data to be prohibited from being deleted. “Specification of photographing direction” means specifying from which direction image data photographed at a predetermined point is to be locked. In this case, the image management unit 105 stores, as image data that satisfies the duplication condition, the predetermined point is included in the photographing range, and the angle of view (camera included in the photographing device 10) is based on the traveling direction corresponding to the photographing range. The image data whose shooting direction is included in the range of the angle of view) may be changed to a state where deletion is prohibited.

The transmitting unit 106 acquires the image data specified by the upload instruction from the collected data DB 100a stored in the storage unit 100 and transmits the image data to the image processing device 20. Further, the receiving unit 104 may receive an upload instruction including the upload target time period from the image processing device 20. In this case, the transmitting unit 106 may acquire, from the collected data DB 100a, image data whose photographing time corresponds to the upload target time zone specified in the upload instruction, and transmit the acquired image data to the image processing device 20. If the upload target time zone is a future time, the transmitting unit 106 may transmit the image data photographed by the photographing unit 103 to the image processing device 20 at the timing when the future time arrives. good.

Further, the receiving unit 104 may receive an upload instruction including an upload target time period and an upload target location from the image processing device 20. In this case, the transmitting unit 106 transmits information such that, among the image data stored in the storage unit 100, the photographing range generated from the photographable range based on the driving position and driving direction in the upload target time period overlaps with the upload target location. Image data that satisfies the conditions may be transmitted to the image processing device 20.

The detection unit 107 detects a wireless signal transmitted from a BLE tag located near the vehicle C. Furthermore, the detection unit 107 associates the ID of the BLE tag included in the detected wireless signal with the detection location and detection time at which the BLE tag was detected, and transmits them to the image processing device 20 .

(Image processing device)
FIG. 6 is a diagram showing an example of the functional block configuration of the image processing device 20. As shown in FIG. The image processing device 20 includes a storage section 200, an acquisition section 201, a reception section 202, an extraction section 203, a transmission section 204, an image detection section 205, and a display control section 206. The storage unit 200 can be realized using the storage device 22 included in the image processing device 20. In addition, the acquisition unit 201 , the reception unit 202 , the extraction unit 203 , the transmission unit 204 , the image detection unit 205 , and the display control unit 206 are the processor 21 of the image processing device 20 stored in the storage device 22 . This can be achieved by running a program. Further, the program can be stored in a storage medium. The storage medium storing the program may be a computer-readable non-transitory storage medium. The non-temporary storage medium is not particularly limited, but may be a storage medium such as a USB memory or a CD-ROM.

The storage unit 200 stores a collected data DB 200a and a detection result DB 200b. The collected data DB 200a stores driving histories including driving positions, driving directions, and times of a plurality of vehicles whose exteriors are photographed.

FIG. 7 is a diagram showing an example of the collected data DB 200a and the detection result DB 200b. The collected data DB 200a stores a driving history DB 200-1a, an image data DB 200-2a, and BLE data 200-3a for each vehicle C. The "vehicle ID" is an identifier that uniquely identifies the vehicle C, and may be, for example, a registration number such as a license plate. The image data DB 100-2a may be a folder that stores image data files. The travel history DB 200-1a stores the travel history of the vehicle C and image data photographed by the photographing device 10. The driving history DB 200-1a and the image data DB 200-2a are the same as the driving history DB 100-1a and the image data DB 100-2a described with reference to FIG. 5, so a description thereof will be omitted. The BLE data 200-3a is a database that stores data that is received from the imaging device 10 and associates the ID of a BLE tag, the detection location and time of detection of the BLE tag.

The detection result DB 200b stores the results of analyzing whether or not the detection target is included in the image data collected from the photographing device 10. “Detection ID” is an identifier that uniquely identifies a detection result. “Type” indicates the type of the detected detection target. "Point information" indicates the point where the detection target is detected. "Photographing time" indicates the time when the detection target was captured in the image data. “Vehicle ID” indicates the vehicle ID of the vehicle C that captured the image data in which the detection target is captured. Returning to FIG. 6, the explanation will be continued.

The acquisition unit 201 acquires running positions, running directions, and times from a plurality of vehicles C at predetermined intervals, associates the acquired running positions, running directions, and times, and stores them in the "driving history" of the collected data DB 200a. do.

The reception unit 202 receives from the user a designation of an upload target location and an upload target time period, or a lock target location and a lock target time slot.

The extraction unit 203 extracts a photographing range generated from a photographable range based on the traveling position and traveling direction of the vehicle C in the upload target time period (or lock target time period) based on the "driving history" of the collected data DB 200a. and the upload target location (or lock target location) satisfy the overlap condition. One or more vehicles C are extracted.

In addition, the reception unit 202 accepts the designation of a predetermined range (for example, 1 km square around the station) as the upload target location (or lock target location), and the extraction unit 203 selects one or more vehicles C that satisfy the overlap condition. One or more vehicles C whose photographic range overlaps with at least a part of the predetermined range may be extracted.

Further, the receiving unit 202 may also accept a designation of a predetermined point (for example, a specific building, etc.) as the upload target location (or lock target location). In this case, the extraction unit 203 may extract one or more vehicles whose shooting range includes the predetermined point as the one or more vehicles that satisfy the overlap condition.

Further, the reception unit 202 may accept designation of a shooting direction and designation of a predetermined point as an upload target location (or lock target location). In this case, the extraction unit 203 selects one or more vehicles C that satisfy the overlap condition, the predetermined point is included in the shooting range, and the angle of view based on the traveling direction corresponding to the shooting range (the shooting device 10 is One or more vehicles whose shooting direction is included in the range of the angle of view of the equipped camera may be extracted.

The transmitting unit 204 sends an upload instruction (or locking) to one or more vehicles extracted by the extracting unit 203 to upload (or lock) image data taken in the upload target time period (or lock target time period). or lock instruction).

Further, the reception unit 202 may accept selection of one or more vehicles for which image data is to be uploaded (or locked) from among the one or more extracted vehicles. In this case, the transmitter 204 may transmit an upload instruction (or lock instruction) to one or more vehicles selected from the one or more extracted vehicles.

Further, the reception unit 202 may accept a designation of a future time as the upload target time period (or lock target time period). In this case, the extraction unit 203 selects one or more vehicles that satisfy the overlap condition after the traveling position and traveling direction associated with the time indicating the future time are stored in the "travel history" of the collected data DB 200a. It may be extracted. Furthermore, the transmitter 204 may transmit an upload instruction (or lock instruction) to one or more extracted vehicles.

Further, each functional unit of the image processing device 20 may perform the following processing in addition to or instead of the processing described above.

The receiving unit 202 receives from the user the designation of the shooting location and shooting time of the image data that is desired to be displayed on the screen of the terminal 30 from among the image data stored in the image processing device 20 .

The extraction unit 203 extracts from the collected data DB 200a that the photographing range generated from the photographing possible range based on the traveling position and traveling direction of the vehicle C in the designated photographing time period and the designated photographing location meet an overlapping condition. Extract image data that satisfies the following.

The display control unit 206 displays the image data extracted by the extraction unit 203.

The image detection unit 205 detects whether or not image data corresponding to the image detection target specified by the user (image data in which the image detection target is captured) exists in the collected data DB 200a. For example, the reception unit 202 receives a designation of an image detection target from the user, and if the image data stored in the collected data DB 200a includes image data corresponding to the image detection target, the image detection unit 205 detects the image detection target. Information indicating the photographing point of the image data corresponding to the image detection target may be associated with the type of the image detection target and recorded in the detection result DB 200b.

The display control unit 206 also causes an icon indicating the type of image detection target recorded in the detection result DB 200b to be displayed on the map screen at a point on the map corresponding to the shooting point of the image data corresponding to the image detection target. You can do it like this.

<Processing procedure>
Next, the "image collection/lock function", "image display function", and "image detection function" provided in the image processing system 1 will be specifically explained. Note that the photographing device 10 mounted on each vehicle C transmits the traveling position, traveling direction, and time of the vehicle C to the image processing device 20 at predetermined intervals (for example, 1 second intervals, 10 second intervals, etc.). It is assumed that the "traveling history" of the collected data DB 200a of the processing device 20 sequentially stores the traveling position, traveling direction, and time transmitted from each photographing device 10.

(Image collection/lock function (Part 1))
FIG. 8 is a flowchart showing the processing procedure performed by the image processing device 20 in the image collection/lock function (part 1).

In step S<b>11 , the reception unit 202 receives instructions from the user regarding the target location and target time period for uploading or locking image data from the user via the screen of the terminal 30 .

FIG. 9 is a diagram showing an example of a vehicle search screen that accepts an instruction to upload or lock image data. The vehicle search screen shown in FIG. 9 includes an input field T100 for inputting the upload or lock target time period, a map display area M100 for specifying the upload or lock target location, and an extracted vehicle list that displays a list of extracted vehicles C. L100 is displayed. The reception unit 202 accepts the time input in the input field T100 as the upload or lock target time. Furthermore, the reception unit 202 accepts the location specified in the map display area M100 as the location to be uploaded or locked. In the example of FIG. 9, nine minutes from 13:16 to 13:25 on December 8th is specified as the upload or lock target time period. It is assumed that the time period to be uploaded or locked is in the past.

Further, the reception unit 202 accepts the designation of a "predetermined range" as a location to be uploaded or locked. In the example of FIG. 9, a predetermined range P100 having a certain size is designated as a location to be uploaded or locked in the map display area M100.

FIG. 10 is a diagram showing an example of a vehicle search screen that accepts an instruction to upload or lock image data. The reception unit 202 accepts the designation of a "predetermined point" as a location to be uploaded or locked, and also receives the designation of a "shooting direction". The shooting direction indicates from which direction the user wants to upload or lock an image taken at a predetermined point. For example, if the user wants to see an image of building A viewed from the south side, the user specifies the location of building A as the predetermined point, and specifies the direction from south to north as the shooting direction. In the example of FIG. 10, in the map display area M100, a point S110 is designated as the location to be uploaded or locked. In addition to specifying the point S110, a photographing direction D110 is specified as the direction in which the point S110 is photographed. Other points not mentioned are the same as FIG. 9. When the vehicle search button B100 in FIGS. 9 and 10 is pressed, the image processing device 20 proceeds to the processing procedure of step S12 in FIG. 8.

In step S12, the extraction unit 203 refers to the "driving history" of the collected data DB 200a, and extracts a photographing range generated from a photographable range based on the vehicle's traveling position and traveling direction during the upload or lock target time period, and One or more vehicles C whose upload or lock target location satisfies the overlap condition are extracted. The processing procedure performed by the extraction unit 203 will be specifically described below.

[Create shooting range]
First, the extraction unit 203 refers to the "traveling history" of the collected data DB 200a, and extracts the traveling position and Get the driving direction. Subsequently, the extraction unit 203 extracts, for each vehicle C, the running position, running direction, direction of the camera mounted on the photographing device 10 (hereinafter referred to as "capable photographing direction"), and camera image during the relevant time period. The photographing range of the photographing device 10 mounted on the vehicle C is calculated based on the angle and the photographable distance of the camera. The angle of view of the camera may be, for example, the horizontal angle of view of a lens included in the photographing device 10. Furthermore, the shooting distance of the camera refers to the distance from the lens position to which it is considered possible to shoot, and is predetermined for each shooting device 10 based on lens performance, number of pixels, etc. It's okay. For example, it may be 100 meters or 200 meters.

FIG. 11 is a diagram for explaining the photographing range. L indicates the photographable distance, a indicates the angle of view, b indicates the traveling direction, and c indicates the photographable direction. The photographable direction c is expressed as a clockwise angle with the traveling direction as a reference (0 degrees). In the example of FIG. 11, the photographing possible direction is the same as the traveling direction of the vehicle C. Normally, in the case of a front camera of a drive recorder, the photographing direction is the same as the traveling direction of the vehicle C (that is, 0 degrees). Furthermore, in the case of a rear camera of a drive recorder, the photographing direction is opposite to the traveling direction of the vehicle C (that is, 180 degrees). In this embodiment, the photographing range when the vehicle C is stationary is referred to as the "photographable range". RE300 in FIG. 11 indicates the photographable range, which can be determined from the traveling direction, the photographable direction c, the angle of view a, and the photographable distance.

Driving route R300 indicates the route traveled by vehicle C during the upload or lock target time period. The driving route R300 can be obtained by arranging the driving positions of the vehicle C in chronological order during the upload or lock target time period (13:16 to 25 minutes on December 8th). The photographing range C300 indicates the range that can be photographed with the photographing device 10 mounted on the vehicle C during the upload or lock target time period. The photographing range C300 can be calculated by moving the photographable range RE300 along the driving route R300. For example, the extraction unit 203 moves the photographable range RE300 along the travel route RE300 from the start point to the end point of the travel route RE300, and determines the range filled with the photographable range RE300 as the photographic range C300. . In other words, the photographing range C300 means the entire range photographed while the vehicle C (photographing device 10) is moving during the upload or lock target time period.

[Vehicle extraction]
Subsequently, the extraction unit 203 extracts one or more vehicles that satisfy the overlap condition by determining for each vehicle C whether the shooting range and the upload or lock target location satisfy the overlap condition. First, a case will be described in which a "predetermined range" is specified as a location to be uploaded or locked. In this case, the extraction unit 203 determines for each vehicle C whether the photographing range and at least a part of the predetermined range overlap.

FIG. 12 is a diagram for explaining the overlap condition. In the specific example E400 of FIG. 12, the photographing range C400 and the predetermined range P400 do not overlap. Therefore, the extraction unit 203 determines that vehicle C does not satisfy the overlap condition. On the other hand, in specific example E410, the photographing range C410 overlaps with a part of the predetermined range P410. Therefore, the extraction unit 203 determines that vehicle C satisfies the overlap condition. Furthermore, in the specific example E410, the photographing range C410 overlaps the entire predetermined range P410. Therefore, the extraction unit 203 determines that vehicle C satisfies the overlap condition.

Next, a case will be described in which the designation of a "predetermined point" and the designation of a "photography direction" are accepted as the location to be uploaded or locked. In this case, the extraction unit 203 extracts, for each vehicle C, an image in which the predetermined point is included in the photographing range and the traveling direction corresponding to the photographing range (that is, the traveling direction at the time when the photographing range was photographed) is the reference point. It is determined whether the shooting direction is included in the corner range. For example, it is assumed that the photographing direction D501, the running direction c, and the range of the angle of view are expressed as clockwise angles when the running direction is set to 0 degrees with respect to the reference direction (for example, the north direction). If the running direction c is 90 degrees and the angle of view is 90 degrees, then the range of the angle of view based on the running direction c is in the range of 45 degrees to 135 degrees. Similarly, if the running direction c is 180 degrees and the angle of view is 90 degrees, the range of the angle of view based on the running direction c is in the range of 135 degrees to 215 degrees.

FIG. 13 is a diagram for explaining the overlap condition. In the specific example E500 of FIG. 13, the predetermined point S500 is included in the photographing range C500, and the photographing direction D501 is included in the range of the angle of view based on the traveling direction c. Therefore, the extraction unit 203 determines that vehicle C satisfies the overlap condition. On the other hand, in the specific example E510 of FIG. 13, the predetermined point S510 is not included in the photographing range C510. Therefore, the extraction unit 203 determines that vehicle C does not satisfy the overlap condition. Further, in the specific example E520 of FIG. 13, the predetermined point S520 is included in the photographing range C520, but the photographing direction D520 is not included in the range of the angle of view based on the traveling direction c. Therefore, the extraction unit 203 determines that vehicle C does not satisfy the overlap condition.

Note that, as described above, the photographing range is the entire range photographed while the vehicle C is moving during the upload or lock target time period. However, since the vehicle C travels on the road, the traveling direction may change during the upload or lock target time period. Therefore, when the driving direction changes during the upload or lock target time period, the extraction unit 203 determines that the vehicle C satisfies the overlap condition if the state of specific example E500 is satisfied in any of the driving directions. It's okay. In other words, if the predetermined point S500 is included in the shooting range C500 and there is a time when the angle of view based on the driving direction c is included in the shooting direction D501 in the upload or lock target time period, the extraction unit 203 extracts , vehicle C may be determined to satisfy the overlapping condition. Depending on the direction in which vehicle C is traveling near a predetermined point, such as when the road on which vehicle C is traveling is curved, it is assumed that a vehicle C that has not photographed the predetermined point from the shooting direction may be extracted. However, at least it is possible to narrow down the vehicles C whose image data is to be uploaded or locked.

The extraction unit 203 extracts one or more vehicles C that satisfy the overlap condition by performing the processing procedure described above. The display control unit 206 displays a list of one or more extracted vehicles C in an extracted vehicle list L100 shown in FIGS. 9 and 10. Returning to FIG. 8, the explanation will be continued.

In step S13, the receiving unit 202 receives a selection of a vehicle (hereinafter referred to as "instruction target vehicle") for which image data upload or locking is to be instructed from among the one or more extracted vehicles. For example, the user can select one or more vehicles from the extracted vehicle list L100 in FIGS. 9 and 10. Note that the processing procedure of step S13 in FIG. 8 may be omitted. In that case, all the vehicles C extracted in the processing procedure of step S12 become instruction target vehicles.

In step S14, the reception unit 202 receives a selection from the user as to whether to instruct the instruction target vehicle to upload the image data or to lock the image data. When the image upload button B101 in FIGS. 9 and 10 is pressed, the reception unit 202 accepts a selection to instruct the instruction target vehicle to upload image data. On the other hand, when the image lock button B102 in FIGS. 9 and 10 is pressed, the reception unit 202 accepts the selection when instructing the instruction target vehicle to lock the image data. If locking is to be instructed, the process proceeds to step S15. When instructing to upload, the process proceeds to step S16.

In step S15, the transmitting unit 204 transmits a lock instruction including the lock target time period received from the user in the processing procedure of step S11 as a lock condition to the instruction target vehicle. When there are a plurality of instruction target vehicles, the transmitting unit 204 transmits a lock instruction including the lock target time period to each of the plurality of vehicles C. For example, in the example shown in FIG. 9, if the image lock button B102 is pressed with December 8th 13:16 to 25 being specified as the target time zone, the transmitting unit 204 determines that the lock target time zone is 12:16 p.m. A lock instruction from 1:16 p.m. to 1:25 p.m. on August 8th is sent to all vehicles subject to the instruction.

In step S16, the transmitting unit 204 transmits, as an upload condition, an upload instruction including the upload target time period received from the user in the processing procedure of step S11 to the instruction target vehicle. When there are a plurality of instruction target vehicles, the transmitter 204 transmits an upload instruction including the upload or lock target time period to each of the plurality of vehicles C. For example, in the example of FIG. 9, if the image upload button B101 is pressed with December 8th 13:16-25 specified as the target time period, the transmitting unit 204 determines that the upload target time period is 12:16 p.m. An upload instruction will be sent to all vehicles subject to the instruction from 1:16 p.m. to 25 p.m. on the 8th of the month.

In step S17, the acquisition unit 201 acquires image data from the vehicle C that transmitted the upload instruction, and stores it in the collected data DB 200a.

In step S18, the display control unit 206 causes the terminal 30 to display the video data stored in the collected data DB 200a.

[Variation example of upload lock target time period]
In the processing procedure of step S15 described above, the transmitter 204 transmits a lock instruction that includes the lock target time period specified by the user as is. In other words, if the user inputs 9 minutes from 1:16 p.m. to 1:25 p.m. on December 8 as the target time period in FIG. 9 or FIG. A lock instruction between 16:00 and 25:00 is sent to the vehicle to be instructed. Similarly, in the processing procedure of step S16, the transmitter 204 transmits an upload instruction whose upload target time period is December 8th from 1:16 p.m. to 2:25 p.m. to all the target vehicles.

However, the target time period may actually include a time period other than the time period when the location to be uploaded or locked is actually shown in the image. Therefore, the image processing device 20 sends an upload or lock instruction by specifying the time period during which the photographing device 10 was actually photographing the upload or lock target location, out of the upload or lock target time periods specified by the user. You may also do so.

FIG. 14 is a diagram for explaining a method of calculating the time period during which the location to be uploaded or locked was photographed. As shown in specific examples E550 and E560, when the predetermined range P550 is specified as the location to be uploaded or locked, the extraction unit 203 extracts the time X when the photographable range RE300 and the predetermined range P550 begin to overlap The time X+t1 at which the possible range RE300 and the predetermined range P550 no longer overlap is specified, and the period from the specified time X to the time X+t1 is set as the time period during which the photographing device 10 was photographing the location to be uploaded or locked. You can. That is, the extraction unit 203 may set the time period in which the photographable range RE300 and the predetermined range P550 overlapped to be the time period in which the photographing device 10 was photographing the location to be uploaded or locked.

Further, as shown in specific examples E570 and E580, when the predetermined point S570 and the shooting direction D570 are specified as the upload or lock target location, the extraction unit 203 detects that the shooting possible range RE300 and the predetermined point S570 start to overlap. time Y and time Y+t2 at which the photographable range RE300 and the predetermined point S570 no longer overlap are specified. Subsequently, the extraction unit 203 determines whether or not there is a time in the time period from time Y to time Y+t2 in which the photographing direction D570 is included in the range of the angle of view based on the traveling direction b. If there is a time when the shooting direction D570 is included in the range of the angle of view based on the running direction b, the shooting device 10 was shooting the location to be uploaded or locked from the specified time Y to the time Y+t2. It may be set as a time zone. If there is no time when the shooting direction D570 is included in the angle of view based on the running direction b, the shooting device 10 does not shoot the location to be uploaded or locked from the specified time Y to the time Y+t2. You may choose not to set it to a specific time period.

This can prevent image data to be uploaded or locked from including image data in which the location to be uploaded or locked has not been photographed.

FIG. 15 is a flowchart showing the processing procedure performed by the photographing device 10 in the image collection/lock function (Part 1).

In step S21, the receiving unit 104 receives a lock instruction including a lock condition or an upload instruction including an upload condition from the image processing device 20 that collects image data from a plurality of vehicles C. If an upload instruction is received, the process proceeds to step S22. If a lock instruction is received, the process proceeds to step S23.

In step S22, the transmitting unit 106 acquires image data that satisfies the upload conditions specified by the upload instruction from the collected data DB 100a stored in the storage unit 100, and transmits it to the image processing device 20. For example, if December 8th from 13:16 to 25 is specified as the upload target time period, the transmitting unit 106 transmits data corresponding to December 8 from 13:16 to 25 from the collected data DB 100a. Image data is acquired and transmitted to the image processing device 20.

In step S23, the image management unit 105 changes the image data that satisfies the lock condition specified by the lock instruction from among the image data stored in the collected data DB 100a to a state in which deletion is prohibited. For example, when 13:16 to 25 on December 8th is specified as the lock target time period, the image management unit 105 selects the image data stored in the collected data DB 100a from 13:16 on December 8th to 13:25 on December 8th. The image data corresponding to 16:00 to 25:00 is changed to a state where deletion is prohibited.

In the above, the image collection/lock function explained using FIGS. 8 to 15 has been described assuming that a past time period is specified as the upload or lock target time period. However, the reception unit 202 of the image processing device 20 can also accept the designation of a future time period as the upload or lock target time period in the processing procedure of step S11 in FIG. In this case, the extraction unit 203 of the image processing device 20 waits until the specified future time period, and adds the traveling position and After the traveling direction is stored, the processing procedure of step S12 in FIG. 8 may be executed.

(Image collection/lock function (Part 2))
In the image collection/lock function (part 1) described above, the image processing device 20 side extracts the instruction target vehicle, and executes an upload instruction or a lock instruction for the extracted vehicle C. On the other hand, in the image collection/lock function (part 2), conditions for uploading image data or conditions for locking image data are instructed to the photographing device 10 of each vehicle C, and the photographing device 10 of each vehicle C Uploads or locks image data by itself if the conditions are met.

FIG. 16 is a flowchart showing the processing procedure performed by the image processing device 20 regarding the image collection/lock function (Part 2).

In step S31, the receiving unit 202 receives instructions for a target location and a target time period from the user via the screen of the terminal 30 as conditions for uploading or locking the image data photographed by the photographing device 10.

For example, the reception unit 202 accepts the time input in the input field T100 in FIG. 9 or 10 as the upload or lock target time period. Further, the reception unit 202 accepts the location specified in the map display area M100 in FIG. 9 or 10 as the location to be uploaded or locked. More specifically, as shown in FIG. 9, when a predetermined range P100 having a certain size is designated as the location to be uploaded or locked, the reception unit 202 selects a location within the predetermined range as the location to be uploaded or locked. Accept specifications.

Further, as shown in FIG. 10, if point S110 is specified as the upload or lock target location, and photographing direction D110 is further specified as the direction for photographing point S110, the receiving unit 202 As the location, it accepts the designation of a predetermined point and also the designation of the shooting direction. Note that a past time may be specified as the upload or lock target time period, or a future time period may be specified. When the user presses the image upload button B101 or the image lock button B102, the process proceeds to step S32 in FIG. 16.

In step S32, the transmitting unit sends, to each vehicle C stored in the "vehicle ID" of the collected data DB 200a, an upload instruction including the upload target time period and upload target location received from the user in the processing procedure of step S31; Alternatively, the lock instruction including the lock target time period and lock target location received from the user in the processing procedure of step S31 is transmitted.

For example, in the example of FIG. 9, if the image lock button B102 is pressed with 13:16 to 25 on December 8th being specified as the target time zone and a predetermined range P100 being specified as the lock target location, The transmitting unit 204 transmits a lock instruction to the vehicle C in which the lock target location is a predetermined range P100 and the lock target time period is December 8th from 13:16 to 13:25. For example, in the example of FIG. 9, if the image upload button B101 is pressed with December 8th 13:16 to 25 being specified as the target time zone, the transmitting unit 204 determines that the upload target location is An upload instruction is sent to vehicle C that is within the predetermined range P100 and that the upload target time period is December 8th from 13:16 to 25 minutes.

FIG. 17 is a flowchart showing the processing procedure performed by the photographing device 10 regarding the image collection/lock function (Part 2). The processing procedure shown in FIG. 17 is repeatedly executed at a predetermined period.

In step S<b>41 , the receiving unit 104 receives/stands by for a lock instruction or an upload instruction from the image processing device 20 .

In step S42, the receiving unit 104 stores the lock condition included in the lock instruction or the upload condition included in the upload instruction received from the image processing device 20 in the instruction content DB 100b. For example, assume that a lock instruction is received in which a predetermined range is specified as the location to be locked and a time period from 0:00 on November 1, 2021 to 23:59 on November 20, 2021 is specified. . In this case, the receiving unit 104 stores the lock condition in the instruction content DB 100b in FIG. 5 by generating a record with the instruction ID J001.

FIG. 18 is a flowchart showing the processing procedure performed by the photographing device 10 regarding the image collection/lock function (Part 2). The processing procedure shown in FIG. 18 is repeatedly executed at a predetermined period.

In step S51, the image management unit 105 searches for image data that satisfies the upload condition or the lock condition among the image data stored in the collected data DB 100a. If it exists, the process proceeds to step S52; if it does not exist, the process ends.

In step S52, the transmitting unit 106 transmits to the image processing device 20, among the image data stored in the collected data DB 100a, image data that satisfies the upload conditions. Furthermore, the image management unit 105 changes the image data that satisfies the lock condition from among the image data stored in the collected data DB 100a to a state in which deletion is prohibited.

Here, the processing procedures of step S51 and step S52 will be specifically explained for a plurality of cases.

[Case 1]
When a past time period is specified as the lock target time period, the image management unit 105 selects image data whose shooting time corresponds to the specified past time period among the image data stored in the collection data DB 100a. may be changed to a state where deletion is prohibited.

Further, when a past time period is specified as the upload target time period, the transmitting unit 106 transmits an image whose shooting time corresponds to the specified past time period among the image data stored in the collected data DB 100a. The data may also be sent to the image processing device 20.

[Case 2]
When a future time period is specified as the time period to be locked, the image management unit 105 waits until the future time period, and when the image data of the future time period is stored in the collected data DB 100a, The data may be changed to a state where deletion is prohibited.

Furthermore, if a future time slot is specified as the upload target time slot, the transmitter 106 waits until the future time slot, and when the image data of the future time slot is stored in the collected data DB 100a, The image data may be transmitted to the image processing device 20.

[Case 3]
When a past time period is specified as the lock target time period and a lock target location is specified, the image management unit 105 determines that the photographing time stored in the collected data DB 100a corresponds to the relevant past time period. Prohibits deletion of image data in which the photographing range generated from the photographing range based on the driving position and driving direction in the past time period and the lock target location satisfy the overlapping condition. The state may be changed. For example, time A to time B is specified as the time period to be locked, and the photographing range generated from the photographable range based on the driving position and direction in the relevant time period and the locking place satisfy the overlap condition. In this case, the image management unit 105 changes the image data from time A to time B out of the image data stored in the collected data DB 100a to a state where deletion is prohibited.

More specifically, when the lock target location is specified as a predetermined range, the image management unit 105 selects at least the shooting range in the specified past time period and the predetermined range as image data that satisfies the overlap condition. Image data that partially overlaps may be changed to a state where deletion is prohibited. In addition, the lock instruction includes designation of the shooting direction as a condition for image data to be prohibited from being deleted, and furthermore, if the lock target location is designation of a predetermined point, the image management unit 105 Image data that meets the conditions includes the specified point in the shooting range in the specified past time period, and the shooting direction is within the range of the angle of view based on the driving direction when shooting the shooting range. The included image data may be changed to a state where deletion is prohibited.

Note that the image management unit 105 may lock the image data of the time period during which the photographing device 10 was actually photographing the lock target location among the time periods designated as the lock target time period. The image management unit 105 extracts the time period in which the photographing device 10 was actually photographing the lock target location using the extraction unit of the image processing device 20 described in “[Modified example of upload lock target time period]” above. 203 may calculate the lock target time period using the same method.

In addition, the process when a past time zone is specified as the upload target time zone and the upload target location is specified is as described in case 3, when the past time zone is specified as the lock target time zone, and, In the explanation of the process when a location to be locked is specified, the word "lock" is replaced with the word "upload", and the word "change to a state that prohibits deletion" is replaced with "send to image processing device 20". The process may be the same as replacing the wording "Yes".

Note that the method for the image management unit 105 or the transmission unit 106 to extract image data that satisfies the duplication condition is the extraction method of the image processing device 20 described using the processing procedure of step S12 in FIG. 9 and FIGS. 11 to 15. The method used by the unit 203 to extract image data that satisfies the overlap condition may be the same.

[Case 4]
When a future time zone is specified as the lock target time zone and a lock target location is specified, the image management unit 105 waits until the future time zone, and stores the image data of the future time zone in the collected data DB 100a. is stored in the collected data DB 100a, and is generated from the photographable range based on the driving position and direction in the past time period, which corresponds to the future time period. Image data whose photographic range and lock target location satisfy an overlapping condition may be changed to a state in which deletion is prohibited. Other points not mentioned may be the same as case 3.

Further, if a future time zone is specified as the upload target time zone and a predetermined range is specified as the upload target location, the transmitting unit 106 waits until the future time zone and stores the collected data DB 100a in the collected data DB 100a. When image data of a zone is stored, the image data corresponding to the future time period stored in the collection data DB 100a can be photographed based on the running position and direction in the past time period. Image data in which the photographing range generated from the range and the target location satisfy an overlapping condition may be transmitted to the image processing device 20. Other points not mentioned may be the same as case 3.

In step S53, the image management unit 105 or the transmitting unit 106 returns to the processing procedure in step S51, unless the process is to be terminated, such as when the photographing device 10 is shut down.

According to the image collection/lock function described above, only the image data that requires confirmation can be uploaded from the photographing device 10 to the image processing device 20. This makes it possible to reduce the processing load on the image processing device 20 and the load on the communication network, compared to the case where images are constantly uploaded from each photographing device 10 to the image processing device 20. In addition, according to the image collection/lock function, by instructing to lock image data for which it is unclear whether confirmation will be required at a later date, the image data can be temporarily saved on the imaging device 10 side, and when confirmation is required. It will be possible to upload to. Therefore, it is possible to further reduce the load on the communication network, which instructs the photographing device 10 to lock image data and instructs to upload it when necessary. In addition, the photographing device 10 locks/uploads the image data by receiving an instruction from the image processing device 20 that collects image data from a plurality of vehicles C, so that the image processing device 20 that collects image data from a plurality of vehicles C and the image processing device 20, it becomes possible to collect appropriate image data at an appropriate timing.

(Image display function)
The image processing device 20 displays the image data stored in the collected data DB 200a on the screen of the terminal 30. Note that the image data stored in the collected data DB 200a may be image data uploaded by the processing procedure shown in FIG. 8. Furthermore, without being limited to the processing procedure shown in FIG. 8, image data uploaded from all vehicles C at a predetermined period (for example, every 10 minutes, every hour, etc.) may be accumulated in the collected data DB 200a. good.

FIG. 19 is a flowchart illustrating an example of a processing procedure when the image processing device 20 displays image data.

In step S61, the reception unit 202 receives a request from the user via the terminal 30 about the location where the image data to be displayed on the screen of the terminal 30 was taken (photographing location) and the image data to be displayed on the screen of the terminal 30. Accepts the specification of the time period in which the photograph was taken (photographing time period).

FIG. 20 is a diagram showing the image display screen (part 1). The image display screen shown in FIG. 20 displays an input field T600 for inputting a shooting time period and a map display area M600 for specifying a shooting location. The reception unit 202 accepts the date and time input in the input field T600 as the shooting time period. Further, the reception unit 202 accepts the location specified in the map display area M600 as a shooting location. In the example of FIG. 20, nine minutes from 13:16 to 13:25 on December 8th is specified as the shooting time period. Note that, unlike the screen shown in FIG. 9, the image display screen is a screen that displays image data that has already been uploaded, so it may be possible to input only past time periods in the input field T600.

Further, the receiving unit 202 receives a designation of a predetermined range as a shooting location. In the example of FIG. 20, a predetermined range P600 having a certain size is designated as a shooting location in the map display area M600. Note that in the map display area M600, it is also possible to specify a predetermined point (for example, a specific building) as a shooting location. In this case, the reception unit 202 accepts the designation of a predetermined point as a shooting location. In addition to specifying the predetermined point, it is also possible to specify the photographing direction in which the predetermined point was photographed. In this case, the reception unit 202 accepts the designation of a predetermined point and the direction in which the predetermined point was photographed. Returning to FIG. 19, the explanation will be continued.

In step S62, the extraction unit 203 determines, from the collected data DB 200a, that the photographing range generated from the photographable range based on the traveling position and traveling direction of the vehicle C during the photographing time period and the photographing location satisfy the overlapping condition. , and extract the image data photographed during the photographing time period.

Regarding the method by which the extraction unit 203 extracts image data from the collected data DB 200a, please refer to the "upload or lock target time period" in the processing procedure of step S12 in FIG. 9 and the processing procedure explained using FIGS. 11 to 15. and "upload or lock target location" may be replaced with "shooting time period" and "shooting location", respectively.

In step S63, the display control unit 206 causes the terminal 30 to display a list of the extracted image data. Further, the receiving unit 202 receives from the user a selection of image data to be displayed from the list. For example, as shown in FIG. 20, the display control unit 206 may display one or more vehicles C that captured the extracted image data in an overlapping manner on the map display area M600. The reception unit 202 may accept a selection of a vehicle C whose image data is to be confirmed from the map display area M600.

In step S64, the display control unit 206 displays the selected image data on the terminal 30. For example, on the image display screen in FIG. 20, the selected image data is displayed in the image display area V600. Furthermore, when the slider S600 is operated by the user, the display control unit 206 displays a still image at the shooting time specified by the slider S600 in the image display area V600.

[Display multiple image data for a specific point]
In the present embodiment, the image processing device 20 receives from the user a designation of a designated point and a designated time as a photographing location and a photographing time period to be displayed on the screen of the terminal 30, and outputs a plurality of image data taken at the designated point. They may be displayed side by side. In this case, in step S61, the receiving unit 202 receives a designation of a designated point (predetermined point) and a designated time (predetermined time) from the user via the terminal 30. In addition, in step S62, the extraction unit 203 extracts first extracted image data from the collected data DB 200a as image data of a time period in which the specified point overlaps with the photographable range based on the traveling position and direction of the vehicle C. and second extracted image data.

More specifically, the extraction unit 203 extracts from the image data stored in the image data DB 200-2a whether the designated point is in a time period where the designated point overlaps the photographable range based on the traveling position and direction of the vehicle C. , image data of a time period including a photographing time before the specified time is extracted as first extracted image data. Further, the extraction unit 203 extracts from the image data stored in the image data DB 200-2a a time period in which the designated point overlaps with the photographable range based on the traveling position and direction of the vehicle C, and from the designated time. Image data of a time period including a later photographing time is extracted as second extracted image data.

Note that the first extracted image data and the second extracted image data may be extracted from image data taken by different vehicles C, or may be extracted from image data taken by the same vehicle C. . For example, the first extracted image data may be extracted from image data taken by vehicle C in the morning at a specific point, and the second extracted image data may be extracted from image data taken by the same vehicle C in the evening.

FIG. 21 is a diagram for explaining an image data extraction method. In specific examples E700 and E710 of FIG. 21, the photographable range RE300 is the same as the photographable range RE300 described with reference to FIG. The extraction unit 203 executes the following procedures A and B for each vehicle C using the travel history and image data of each vehicle C stored in the collected data DB 200a. Furthermore, after procedures A and B are completed for all vehicles C, the extraction unit 203 executes procedure C and subsequent steps.

"Procedure A": The extraction unit 203 refers to the "driving history" and determines whether or not there is a time period in which the specified point S700 overlaps in the photographable range RE300 (hereinafter referred to as "overlapping time period"). judge. If it exists, proceed to "Procedure B"; if it does not exist, the process ends.

"Procedure B": The extraction unit 203 identifies the time period from the time Z when the photographable range RE300 and the specified point S700 start to overlap to the time Z+t3 when the designated point S700 leaves the photographable range RE300 as the overlapping time period.

"Procedure C": The extraction unit 203 determines whether there is one or more vehicles C whose time period is earlier than the designated time in the overlapping time period specified in step B, and the overlapping time period specified in step B. It is determined whether there is one or more vehicles C whose time is later than the specified time. If it exists, proceed to step D; if it does not exist, the process ends.

“Procedure D”: The extraction unit 203 extracts any one vehicle C (hereinafter referred to as (referred to as "first vehicle"). Also, any one vehicle C (hereinafter referred to as the "second vehicle") from among the one or more vehicles C whose time after the designated time is included in the overlapping time period identified in step B Select.

“Procedure E”: The extraction unit 203 extracts image data taken in a time period from time Z to time Z+t3 from the image data stored in the image data DB 200-2a of the first vehicle as first extracted image data. Extract as. Further, the extraction unit 203 extracts image data photographed in a time period from time Z to time Z+t3 as second extracted image data from the image data stored in the image data DB 200-2a of the second vehicle. .

FIG. 22 is a diagram for explaining an image data extraction method. First, the extraction unit 203 identifies overlapping time periods for each of vehicles C1 to C4 by executing procedure A and procedure B. Assume that there is no overlapping time period for vehicle C5. Subsequently, the extraction unit 203 executes procedure C. According to FIG. 22, the vehicles C whose overlapping time period includes a time before the specified time are vehicle C1, vehicle C3, and vehicle C4, and the vehicles whose overlapping time period includes a time after the specified time. C is vehicle C1, vehicle C2, and vehicle C4. Therefore, the extraction unit 203 proceeds to step D.

In step D, the extraction unit 203 selects the first vehicle C from among vehicles C2, C3, and C4, and selects the second vehicle C from among vehicles C1 and C2. The selection method is arbitrary, but for example, vehicles C (vehicles C1 and C4 in FIG. 22) whose designated time is included in the overlapping time period may be selected, or vehicles whose designated time is not included in the overlapping time period may be selected. C (vehicle C2 and vehicle C3 in FIG. 22) may be selected. Here, it is assumed that the vehicle C2 is selected as the first vehicle C and the vehicle C3 is selected as the second vehicle C. Subsequently, in step E, the extraction unit 203 extracts first extracted image data from the image data of vehicle C2, and extracts second extracted image data from the image data of vehicle C3.

For example, if the overlapping time period of vehicle C2 is from 13:19:30 to 13:20:00, the extraction unit 203 extracts from the image data of vehicle C2 that the photographing time is from 13:19:30 to 13:00. Image data of 20 minutes 00 seconds is extracted as first extracted image data. Further, when the overlapping time period of vehicle C3 is from 13:34:45 to 13:35:10, the extraction unit 203 extracts the photographing time from the image data of vehicle C3 from 13:34:45 to 13:00. Image data of 35 minutes and 10 seconds is extracted as second extracted image data.

FIG. 23 is a diagram showing the image display screen (Part 2). The image display screen shown in FIG. 23 displays an input field T800 for inputting a specified time and a map display area M800 for specifying a shooting location. In the example of FIG. 23, it is assumed that a designated point (point indicated by S850) is designated as the shooting location, and 13:30:00 is designated as the designated time. Further, the first extracted image data is image data whose photographing time is from 13:19:30 to 13:20:00, and the second extracted image data is image data whose photographing time is from 13:34:45 to 13:00. It is assumed that the image data is 35 minutes and 10 seconds.

As shown in FIG. 23, the display control unit 206 displays the first extracted image data in the image display area V800 and displays the second extracted image data in the image display area V810. At this time, the display control unit 206 displays a still image (first still image) taken at a time before the specified time for the first extracted image data, and displays a still image taken at a time earlier than the specified time for the second extracted image data. A still image taken at a later time (second still image) may be displayed.

In the example of FIG. 23, the image display area V800 displays a still image taken at 13:19:40, which is before the designated time (13:30:00), and the image display area V810 displays the following: A still image captured at 13:35:00, which is after the designated time (13:30:00), is displayed. Furthermore, when the slider S800 is operated, a still image at the shooting time specified by the slider S800 is displayed. When the slider S810 is operated, a still image at the shooting time specified by the slider S800 is displayed. As a result, on the image display screen, still images taken at different times (for example, before and after the specified time) of the shooting location are displayed side by side, so the user can, for example, see the time when an accident occurred. As a reference, it becomes possible to check whether there is a change over time in the same place.

The display control unit 206 also indicates, in the map display area M800, the running position and running direction of the vehicle C at the time when the image data displayed in the image display area V800 was photographed (that is, 13:19:40). Display icon N800. Similarly, the display control unit 206 displays, in the map display area M810, the running position and running direction of the vehicle C at the time when the image data displayed in the image display area V810 was photographed (that is, 13:35:00). An icon N810 is displayed. This allows the user to understand where the image data was taken.

In addition, the extraction unit 203 extracts information such that the traveling direction of the vehicle C at the time when the first image data is photographed and the traveling direction of the vehicle C at the time when the second image data is photographed are within a predetermined angular range (for example, 30 degrees). The first image data and the second image data may be extracted so that they are within the same range. For example, assume that the extraction unit 203 has extracted image data from 13:19:30 to 13:20:00 as the first image data. In this case, the extraction unit 203 captures the traveling direction of the vehicle C that captured the first image data in the time period from 13:19:30 to 13:20:00, and the image data to be extracted as the second image data. The second image data is extracted such that the traveling direction of the vehicle C falls within a predetermined angle range. When checking for changes over time in the same place, it is better to compare images of the same place taken from the same direction than to compare images taken of the same place taken from different directions. It is considered easy to confirm. This allows the user to easily check whether there is a change over time at the same location.

(Image detection function)
The image processing device 20 analyzes the image data collected from the photographing device 10 to detect a detection target appearing in the image data. Note that the image data stored in the collected data DB 200a may be image data uploaded according to the processing procedure shown in FIG. 8. Furthermore, without being limited to the processing procedure shown in FIG. 8, image data uploaded from all vehicles C at a predetermined period (for example, every 10 minutes, every hour, etc.) may be accumulated in the collected data DB 200a. good.

FIG. 24 is a flowchart illustrating an example of a processing procedure when the image processing device 20 performs image detection.

In step S71, the receiving unit 202 receives a designation of an image detection target from the user via the terminal 30. The image detection target may be divided into types such as a specific person, a specific vehicle number, water damage, and cracks, for example. When accepting a specific person as an image detection target, the accepting unit 202 may accept that the type of detection target is a specific person and an image of the specific person. When accepting a specific vehicle number as an image detection target, the reception unit 202 may accept that the type of the detection target is a vehicle number and the number of the vehicle C to be detected. When accepting water damage as an image detection target, the reception unit 202 may accept that the type of detection target is "water damage." When accepting a crack as an image detection target, the accepting unit 202 may accept that the type of detection target is "crack".

In step S72, the image detection unit 205 searches the collected data DB 200a to determine whether image data corresponding to the image detection target specified in step S71 (image data in which the image detection target is captured) exists. If an image corresponding to the image detection target is found, the process proceeds to step S73. For example, the image detection unit 205 determines whether image data corresponding to the image detection target exists by inputting the image data stored in the collected data DB 200a into a trained model that has the ability to identify the image to be detected. Alternatively, a search may be made to determine whether or not.

More specifically, when the image detection target is a specific person, the image detection unit 205 applies image data stored in the collected data DB 200a to a trained model that has the ability to identify the person in the input image. By inputting an image of a specific person received from the user, it may be searched whether or not image data corresponding to the image detection target exists. Further, when the image detection target is a specific vehicle number, the image detection unit 205 uses the image data stored in the collected data DB 200a and the vehicle received from the user in a trained model that has the ability to identify the vehicle number. By inputting a number, it may be searched whether or not image data corresponding to the image detection target exists. Further, when the image detection target is a flood damage, the image detection unit 205 inputs the image data stored in the collected data DB 200a to a trained model that has the ability to identify images of flood damage. Alternatively, a search may be made to determine whether or not image data corresponding to the above exists. Further, when the image detection target is a crack in the road, the image detection unit 205 inputs the image data stored in the collected data DB 200a to a trained model that has the ability to identify images of cracks. A search may be made to determine whether or not image data corresponding to the detection target exists. Furthermore, the method is not limited to using a trained model, and known image detection techniques such as template matching may also be used.

In step S73, the image detection unit 205 adds a tag to the image data in which the image detection target has been detected. More specifically, the image detection unit 205 determines the type of image detection target, the location where the image detection target was photographed, the time when the image detection target was photographed, and the location of the photographing device 10 that photographed the image data. The vehicle ID of vehicle C is stored in the detection result DB 200b.

In step S74, the image detection unit 205 repeatedly executes the processing procedure of steps S72 and S73 until the search ends.

FIG. 25 is a diagram illustrating an example of a screen displaying detected image detection targets. As shown in FIG. 25, the display control unit 206 causes an icon corresponding to the type of image detection target recorded in the detection result DB 200b to be displayed at the shooting point of the image that is the image detection target on the map screen. Good too. Icons TA900 and TA901 indicate locations where cracks in the road have been detected. Icon TA910 indicates the location where a particular person was detected. Icons TA920, TA921, and TA922 indicate locations where water damage has been detected.

Note that the image processing device 20 may have a function of detecting a BLE tag in addition to the detection target shown in the image collected from the imaging device 10. For example, the detection unit 107 of the photographing device 10 may detect BLE tags existing around the vehicle C by searching for beacons transmitted from the BLE tags while the vehicle C is traveling. . When detecting a BLE tag, the transmitting unit 106 of the imaging device 10 may transmit information about the detected BLE tag (for example, BLE tag ID, detection location, and detection time) to the image processing device 20. The image processing device 20 stores the received BLE tag information in "BLE data" of the collected data DB 200a.

Further, the image processing device 20 may accept a designation of a BLE tag to be detected (for example, a BLE tag ID to be detected) in the processing procedure of step S71 in FIG. 24 . The image detection unit 205 searches whether the specified BLE tag exists in “BLE data” of the collected data DB 200a, and if the specified BLE tag exists, the display control unit 206 searches the “BLE data” of the collected data DB 200a. An icon indicating the location where the BLE tag has been detected may be displayed.

According to the image detection function described above, a detection target appearing in an image can be detected using image data collected from the photographing device 10. This makes it possible, for example, to detect cracks on a road or discover events such as flooding in a river.

<Additional information>
The embodiments described above are intended to facilitate understanding of the present invention, and are not intended to be interpreted as limiting the present invention. The flowcharts, sequences, and elements included in the embodiments, as well as their arrangement, materials, conditions, shapes, sizes, etc., described in the embodiments are not limited to those illustrated and can be changed as appropriate. Further, it is possible to partially replace or combine the structures shown in different embodiments.

DESCRIPTION OF SYMBOLS 1... Image processing system, 10... Photographing device, 11... Processor, 12... Storage device, 13... Communication IF, 14... Sensor, 15... Input device, 16... Output device, 17... Camera, 20... Image processing device, 21 ...Processor, 22...Storage device, 23...Communication IF, 24...Input device, 25...Output device, 30...Terminal, 100...Storage unit, 100a...Collected data DB, 100b...Instruction content DB, 101...Measurement unit, 102 ...Notification section, 103...Photography section, 104...Reception section, 105...Image management section, 106...Transmission section, 107...Detection section, 200...Storage section, 200a...Collected data DB, 200b...Detection result DB, 201...Acquisition unit, 202...reception unit, 203...extraction unit, 204...transmission unit, 205...image detection unit, 206...display control unit

Claims (9)

A photographing device mounted on a vehicle,
a photographing unit that photographs the exterior of the vehicle and stores the photographed image data in a storage unit;
a receiving unit that receives a lock instruction that instructs a condition for image data that is prohibited from being deleted from an information processing device that collects image data from a plurality of vehicles including the vehicle;
an image management unit that changes image data that satisfies the condition from among the image data stored in the storage unit to a state where deletion is prohibited when the lock instruction is received;
A photographic device with The photographing unit stores the photographed image data in association with a photographing time in the storage unit,
The lock instruction includes the specification of a target time period as the condition,
The image management unit changes image data whose photographing time corresponds to the specified target time period as image data that satisfies the condition to a state where deletion is prohibited.
The imaging device according to claim 1. a measurement unit that measures the running position and running direction of the vehicle,
The photographing unit stores the photographed image data, the traveling position, and the traveling direction in association with each other in the storage unit,
The lock instruction further includes designation of a target location as the condition,
The image management unit is configured to prohibit deletion of image data in which the target location satisfies an overlapping condition with a photographing range generated from a photographable range based on the traveling position and the traveling direction in the target time period. change to,
The imaging device according to claim 2. The specification of the target location is the specification of a predetermined range,
The image management unit changes image data in which the photographing range and at least a part of the predetermined range overlap, to a state in which deletion is prohibited, as image data that satisfies the duplication condition.
The imaging device according to claim 3. The lock instruction includes designation of a shooting direction as the condition;
The designation of the target location is the designation of a predetermined point,
The image management unit stores, as image data that satisfies the duplication condition, the predetermined point is included in the photographing range, and the image data is stored in a range of angles of view based on the traveling direction corresponding to the photographing range. Change image data that includes orientation to a state that prohibits deletion.
The imaging device according to claim 3 or 4. a notification unit that notifies an information processing device of the running position and running direction of the vehicle at predetermined intervals;
The imaging device according to any one of claims 3 to 5. The receiving unit receives, from the information processing device, an upload instruction instructing to upload image data in a target time period,
a transmitting unit configured to acquire image data whose photographing time corresponds to a target time period specified by the upload instruction from the storage unit and transmit the acquired image data to the information processing device;
The imaging device according to any one of claims 1 to 6. A photographing method performed by a photographing device mounted on a vehicle, the method comprising:
photographing the exterior of the vehicle and storing the photographed image data in a storage unit;
receiving, from an information processing device that collects image data from a plurality of vehicles including the vehicle, a lock instruction that instructs conditions for image data that are prohibited from being deleted;
When the lock instruction is received, changing image data that satisfies the condition from among the image data stored in the storage unit to a state where deletion is prohibited;
including how to shoot. The photographic device installed in the vehicle,
photographing the exterior of the vehicle and storing the photographed image data in a storage unit;
receiving, from an information processing device that collects image data from a plurality of vehicles including the vehicle, a lock instruction that instructs conditions for image data that are prohibited from being deleted;
When the lock instruction is received, changing image data that satisfies the condition from among the image data stored in the storage unit to a state where deletion is prohibited;
A program to run.

Images