JP7041117B2 - Judgment method, computer, output device and system - Google Patents

Description

The present invention relates to a danger determination method, a danger determination device, a danger output device, and a danger determination system.

When the user is moving with a moving object (for example, a vehicle), the dangerous event is notified or warned in advance to the user at a point where a dangerous event such as an accident or a hiyari hat is likely to occur. A danger judgment system that prevents the occurrence is known. In such a danger determination system, a point where a dangerous event is likely to occur (that is, a point with a high degree of danger) is specified based on the detection result of the position, behavior, driving operation, etc. of the moving object, and the specified point is specified. Is stored in association with the map data.

For example, in the danger determination system of Patent Document 1, the danger for each registered point is analyzed by analyzing the image around the moving body acquired by the camera and the position sensor mounted on the moving body and the position information obtained by taking the image. Register the event in the database. When a user passes through a registration point with a moving object, the degree of danger corresponding to the registration point is determined based on the danger event at the registration point registered in the database, and the danger is notified to the user. Warning.

Japanese Unexamined Patent Publication No. 2014-154004

However, in the danger determination system of Patent Document 1, when a user passes through a point other than the registered point registered in the database as a moving object, the risk level corresponding to the point cannot be determined. It is not possible to notify or warn a person of danger.

Therefore, the present invention provides a danger determination method, a danger determination device, a danger output device, and a danger determination system capable of accurately determining the degree of danger.

The danger determination method according to one aspect of the present invention is a danger determination method used in a danger determination system that manages the degree of danger corresponding to a point where a moving object is located, and the danger determination system has generated a danger event. It is equipped with a danger occurrence information management unit that stores one or more danger occurrence information in which danger point information indicating a danger point and first sensor information indicating the occurrence status of the danger event at the danger point are associated with each other. , The danger determination method is a step of acquiring moving body information in which the current position information indicating the current position of the moving body and the second sensor information indicating the peripheral situation of the moving body at the current point are associated with each other. The danger occurrence information corresponding to the current location indicated by the current location information included in the moving body information is not stored in the danger occurrence information management unit, and corresponds to a location other than the current location. When the danger occurrence information is stored in the danger occurrence information management unit, the second sensor information included in the moving body information and one or more of the danger occurrence information storage units are stored in the danger occurrence information management unit. The step includes a step of determining the degree of danger corresponding to the current position based on the similarity with the first sensor information included in the generation information.

It should be noted that these comprehensive or specific embodiments may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program or a computer-readable CD-ROM, and the system, a method, an integrated circuit, or a computer program. And may be realized by any combination of recording media.

According to the danger determination method according to one aspect of the present invention, the degree of danger can be determined with high accuracy.

It is a block diagram which shows the structure of the risk determination system which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the server apparatus which concerns on Embodiment 1. FIG. It is a figure which shows an example of the mobile body information which concerns on Embodiment 1. FIG. It is a figure which shows an example of the danger occurrence information which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the mobile apparatus which concerns on Embodiment 1. FIG. It is a sequence diagram which shows the flow of operation of the danger determination system which concerns on Embodiment 1. FIG. It is a figure for demonstrating an example of the service to which the danger determination system which concerns on Embodiment 1 is applied. It is a block diagram which shows the structure of the risk determination system which concerns on Embodiment 2. FIG. It is a block diagram which shows the structure of the server apparatus which concerns on Embodiment 2. FIG. It is a figure which shows an example of the danger determination information which concerns on Embodiment 2. FIG. It is a sequence diagram which shows the flow of operation of the danger determination system which concerns on Embodiment 2. FIG.

The danger determination method according to one aspect of the present invention is a danger determination method used in a danger determination system that manages the degree of danger corresponding to a point where a moving object is located, and the danger determination system has generated a danger event. It is equipped with a danger occurrence information management unit that stores one or more danger occurrence information in which danger point information indicating a danger point and first sensor information indicating the occurrence status of the danger event at the danger point are associated with each other. , The danger determination method is a step of acquiring moving body information in which the current position information indicating the current position of the moving body and the second sensor information indicating the peripheral situation of the moving body at the current point are associated with each other. The danger occurrence information corresponding to the current location indicated by the current location information included in the moving body information is not stored in the danger occurrence information management unit, and corresponds to a location other than the current location. When the danger occurrence information is stored in the danger occurrence information management unit, the second sensor information included in the moving body information and one or more of the danger occurrence information storage units are stored in the danger occurrence information management unit. The step includes a step of determining the degree of danger corresponding to the current position based on the similarity with the first sensor information included in the generation information.

According to this aspect, even if the danger occurrence information corresponding to the current position indicated by the current position information included in the moving body information is not stored in the danger occurrence information management unit, the second one included in the moving body information. The degree of danger is determined based on the similarity between the sensor information of the above and the first sensor information included in the danger occurrence information corresponding to the point other than the current point. As a result, even when the moving object passes through a point other than the danger point corresponding to the danger occurrence information stored in the danger occurrence information management unit, the danger level can be accurately determined, for example, the moving object. It is possible to notify or warn the user of the danger.

For example, in the determination step, the danger occurrence information corresponding to the current location indicated by the current location information included in the moving body information and the danger occurrence information corresponding to a point other than the current location are the dangers. When stored in the generation information management unit, the second sensor information included in the moving object information and the first sensor information included in the danger occurrence information corresponding to a point other than the current point. It may be configured to determine the degree of risk corresponding to the current location based on the similarity.

According to this aspect, when the danger occurrence information corresponding to the current location and the danger occurrence information corresponding to the location other than the current location are stored in the danger occurrence information management unit, the second is included in the moving object information. When the similarity between the sensor information of 2 and the first sensor information included in the danger occurrence information corresponding to the point other than the current point is the highest, for example, the danger occurrence information corresponding to the point other than the current point is used. To judge the degree of danger. As a result, the degree of danger can be determined more accurately.

For example, the danger occurrence information further includes the severity of the danger event associated with the danger point information and the first sensor information, and in the determination step, the said moving body information is included. The similarity between the second sensor information and the first sensor information included in one or more of the danger occurrence information stored in the danger occurrence information management unit is obtained, and the similarity and the first sensor information are obtained. It may be configured to determine the risk level corresponding to the current position based on the severity level associated with the sensor information.

According to this aspect, since the risk level is determined based on the similarity and the severity level associated with the first sensor information, the risk level can be determined more accurately.

For example, the first sensor information is a first image of the occurrence situation of the dangerous event at the dangerous point, and the second sensor information is an image of the surrounding situation of the moving body at the current point. In the determination step, the risk level corresponding to the current position is determined based on the similarity between the first image and the second image. May be good.

According to this aspect, when the similarity between the first image and the second image is high, it can be estimated that the similarity between the occurrence situation of the dangerous event and the surrounding situation of the moving object at the current position is high. Therefore, by determining the degree of danger based on the similarity between the first image and the second image, the degree of danger can be determined with high accuracy.

For example, the danger determination method further analyzes the first traffic condition at the danger point when the danger event occurs based on the first image, and is based on the second image. In the step of determining, the step of analyzing the second traffic condition at the current location includes the step of analyzing the first traffic condition and the second traffic condition based on the similarity of the analyzed first traffic condition to the current location. It may be configured to determine the corresponding degree of risk.

According to this aspect, when the similarity between the first traffic condition and the second traffic condition is high, it can be estimated that the possibility of a dangerous event occurring at the current location is high. Therefore, by determining the degree of danger based on the similarity between the first traffic condition and the second traffic condition, the degree of danger can be determined accurately.

For example, in the danger determination method, when the danger level corresponding to the determined current location is within a predetermined range, the current location information included in the moving object information is used as the danger location information, and , A step of generating the danger occurrence information using the second sensor information included in the moving body information as the first sensor information, and a step of adding the generated danger occurrence information to the danger occurrence information management unit. And may be configured to include.

According to this aspect, when the degree of danger is within a predetermined range, the moving object information is added to the danger occurrence information management unit as the danger occurrence information, so that the accumulation amount of the danger occurrence information in the danger occurrence information management department is increased. Can be made to.

For example, in the acquisition step, the moving body information transmitted from the moving body device mounted on the moving body is received via the network, and the danger determination method further corresponds to the determined current position. When the risk level is within a predetermined range, the mobile device may be configured to include a step of transmitting risk level information regarding the risk level to the mobile device that has transmitted the mobile body information.

According to this aspect, when the degree of danger is within a predetermined range, the degree of danger information is transmitted to the moving body device that has transmitted the moving body information, so that the user of the moving body passing through the current position is notified. It is possible to notify or warn of danger.

For example, in the acquisition step, the moving body information transmitted from the moving body device mounted on the moving body is received via the network, and the danger determination method further corresponds to the determined current position. When the degree of danger is within a predetermined range, and within a predetermined time after receiving the moving body information, another moving body is included in the moving body information. It may be configured to include a step of transmitting risk information regarding the risk to a mobile device mounted on the other mobile when it is determined to pass the point.

According to this aspect, when the degree of danger is within a predetermined range, the degree of danger information is transmitted to a moving body device different from the moving body device that has transmitted the moving body information. Therefore, for example, to a user of a following vehicle. It is possible to notify or warn of danger.

For example, the danger determination method further includes a step of storing the danger determination information in which the danger level corresponding to the determined current location and the current location information are associated with each other in the danger determination information management unit. In the step, when the danger determination information corresponding to the current location indicated by the current location information included in the moving object information is stored in the danger determination information management unit, the danger included in the danger determination information is stored. It may be configured to use degrees.

According to this aspect, when the danger judgment information corresponding to the current position is stored in the danger judgment information management unit, the danger degree included in the danger judgment information is used, so that the danger degree judgment process can be performed in a relatively short time. Can be done at.

For example, the moving body information further includes a third sensor information indicating a traveling state of the moving body, which is associated with the current position information and the second sensor information, and in the step of determining the determination. The similarity between the second sensor information included in the moving body information and the first sensor information included in one or more of the danger occurrence information stored in the danger occurrence information management unit is obtained. Based on the similarity and the third sensor information, the risk level corresponding to the current position may be determined.

According to this aspect, since the risk level is determined based on the similarity and the third sensor information, the risk level can be determined more accurately.

Further, the danger determination device according to one aspect of the present invention is a danger determination device that manages the degree of danger corresponding to the point where the moving body is located, and includes the danger point information indicating the danger point where the danger event has occurred and the above-mentioned danger point information. The danger occurrence information management unit that stores one or more danger occurrence information associated with the first sensor information indicating the occurrence status of the danger event at the danger point, and the current position indicating the current position of the moving body. A receiving unit that receives the moving body information in which the information is associated with the second sensor information indicating the surrounding situation of the moving body at the current position, and the current position information included in the moving body information indicates the above. The danger occurrence information corresponding to the current position is not stored in the danger occurrence information management unit, and the danger occurrence information corresponding to a point other than the current location is stored in the danger occurrence information management unit. In the case, the similarity between the second sensor information included in the moving body information and the first sensor information included in one or more of the danger occurrence information stored in the danger occurrence information management unit. Based on this, a determination unit for determining the degree of danger corresponding to the current location is provided.

According to this aspect, even if the danger occurrence information corresponding to the current position indicated by the current position information included in the moving body information is not stored in the danger occurrence information management unit, the second one included in the moving body information. The degree of danger is determined based on the similarity between the sensor information of the above and the first sensor information included in the danger occurrence information corresponding to the point other than the current point. As a result, even when the moving object passes through a point other than the danger point corresponding to the danger occurrence information stored in the danger occurrence information management unit, the danger level can be accurately determined, for example, the moving object. It is possible to notify or warn the user of the danger.

Further, the danger determination system according to one aspect of the present invention is a danger determination system including a danger determination device and a danger output device mounted on the moving body, and managing the degree of danger corresponding to the point where the moving body is located. Therefore, the danger determination device provides danger occurrence information in which the danger point information indicating the danger point where the danger event has occurred and the first sensor information indicating the occurrence status of the danger event at the danger point are associated with each other. The danger occurrence information management unit that stores one or more, the current position information indicating the current position of the moving body, and the second sensor information indicating the surrounding situation of the moving body at the current position are associated with each other. The first receiving unit that receives the moving body information and the danger occurrence information corresponding to the current position indicated by the current position information included in the moving body information are not stored in the danger occurrence information management unit. Moreover, when the danger occurrence information corresponding to a point other than the current position is stored in the danger occurrence information management unit, the second sensor information included in the moving body information and the danger occurrence information management unit. A determination unit for determining the degree of danger corresponding to the current position based on the similarity with the first sensor information included in the one or more pieces of the danger occurrence information stored in the above, and the degree of danger. The danger output device includes a first transmission unit that transmits danger level information to the danger output device, and the danger output device includes a second transmission unit that transmits the moving object information to the danger determination device and the danger determination device. A second receiver that receives the risk information transmitted from, and an output that outputs attention information for calling attention to the user of the moving object based on the received risk information. It has a department.

According to this aspect, even if the danger occurrence information corresponding to the current position indicated by the current position information included in the moving body information is not stored in the danger occurrence information management unit, the second one included in the moving body information. The degree of danger is determined based on the similarity between the sensor information of the above and the first sensor information included in the danger occurrence information corresponding to the point other than the current point. As a result, even when the moving object passes through a point other than the danger point corresponding to the danger occurrence information stored in the danger occurrence information management unit, the danger level can be accurately determined, for example, the moving object. It is possible to notify or warn the user of the danger.

Further, the danger output device according to one aspect of the present invention is a danger output device used in the above-mentioned danger determination system, and includes current position information indicating the current position of the moving body and the periphery of the moving body at the current position. A second transmitting unit that transmits the moving object information associated with the second sensor information indicating the situation to the danger determination device, and a second receiving unit that receives the danger level information transmitted from the danger determination device. And an output unit that outputs the attention-calling information for calling the attention to the user of the moving body based on the received danger level information.

It should be noted that these comprehensive or specific embodiments may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program or a computer-readable CD-ROM, and the system, a method, an integrated circuit, or a computer program. Alternatively, it may be realized by any combination of recording media.

Hereinafter, embodiments will be specifically described with reference to the drawings.

It should be noted that all of the embodiments described below show comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, the order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present disclosure. Further, among the components in the following embodiments, the components not described in the independent claim indicating the highest level concept are described as arbitrary components.

(Embodiment 1)
[1-1. Overall configuration of danger judgment system]
The overall configuration of the danger determination system 10 according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing a configuration of the danger determination system 10 according to the first embodiment.

As shown in FIG. 1, the danger determination system 10 includes a server device 101 (an example of a danger determination device) and a plurality of mobile devices 102 (an example of a danger output device).

The server device 101 includes an image database 103 (an example of an image management unit). One or more danger occurrence information is managed (stored) in the image database 103. The danger occurrence information is a danger point image (first sensor information and a first sensor information) which is an image of the surrounding situation (that is, the occurrence situation of the danger event) of the moving body 105 that has encountered a dangerous event such as an accident or a hiyari hat at the danger point. 1) and incidental information (an example of danger point information indicating a danger point) such as the shooting date and time, shooting position and shooting direction of this danger point image (that is, the traveling direction of the moving body 105 that encountered a dangerous event). ) Is the data associated with it. It should be noted that i) a method of shooting an image with various cameras, ii) a method of acquiring a shooting date and time, a shooting position, a shooting direction, etc., and iii) a method of uploading the shot image to the server device 101 are arbitrarily disclosed. The method described above may be used, and the description thereof will be omitted here.

Each of the plurality of mobile devices 102 is communicatively connected to the server device 101 via the network 104, and is mounted on the corresponding mobile device 105. The mobile body 105 is, for example, a vehicle driven by a user. The mobile device 102 transmits the mobile information to the server device 101 via the network 104. The moving body information is an image of the current position (an example of the second sensor information and the second image) which is an image of the surrounding situation of the moving body 105 at the current position, and accompanying information (moving body 105) related to the current position image. It is the data associated with the current location information (an example) indicating the current location of.

The server device 101 analyzes the traffic condition (an example of the second traffic condition) at the current location at the time of taking the current location image based on the received current location image and its accompanying information, and also analyzes the image database 103. The traffic condition (an example of the first traffic condition) at the dangerous point when a dangerous event occurs is analyzed based on the dangerous point image and its accompanying information managed in. The server device 101 determines the risk level of the situation in which the moving body 105 is placed at the time of taking the current position image based on the similarity between the two analyzed traffic conditions, and moves the risk level information regarding the determined risk level. It is transmitted to the body device 102 via the network 104. The degree of danger is an index indicating the possibility that a dangerous event may occur at the current position where the moving body 105 is located at the time of taking the image of the current position.

The mobile device 102 outputs attention information for calling attention to the user of the mobile 105 (or the mobile device 102) based on the received risk information.

[1-2. Server device configuration]
[1-2-1. Overall configuration of server device]
Next, the overall configuration of the server device 101 will be described in detail with reference to FIGS. 2 to 4. FIG. 2 is a block diagram showing a configuration of the server device 101 according to the first embodiment. FIG. 3 is a diagram showing an example of mobile information according to the first embodiment. FIG. 4 is a diagram showing an example of danger occurrence information according to the first embodiment.

As shown in FIG. 2, the server device 101 includes a transmission / reception unit 201 (an example of a first transmission unit and a first reception unit), a danger occurrence information management unit 202, an image analysis unit 203, a risk level determination unit 204, and a control unit. The unit 205 is provided.

The server device 101 includes a microprocessor, a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk, and the like, which are not specifically shown. Computer programs are stored in these RAMs, ROMs, and hard disks. When the microprocessor operates according to this computer program, the server device 101 fulfills its function.

Each functional block of the server device 101, such as the transmission / reception unit 201, the danger occurrence information management unit 202, the image analysis unit 203, the risk level determination unit 204, and the control unit 205, is typically an integrated circuit LSI (Large Scale). It is realized as Information). These may be individually integrated into one chip, or may be integrated into one chip so as to include one or more functional blocks or a part of each functional block. Although it is referred to as an LSI here, it may be referred to as an IC (Integrated Circuit), a system LSI, a super LSI, or an ultra LSI depending on the degree of integration. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. After manufacturing the LSI, an FPGA (Field Programmable Gate Array) that can be programmed or a reconfigurable processor that can reconfigure the connection and setting of the circuit cells inside the LSI may be used. Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. The application of biotechnology may be possible. Finally, each functional block of the server device 101 may be realized by software or may be realized by a combination of LSI and software. Also, the software may be tamper resistant.

[1-2-2. Transmitter / receiver]
The transmission / reception unit 201 receives the mobile information transmitted from the mobile device 102 via the network 104.

Here, an example of mobile information received by the transmission / reception unit 201 will be described with reference to FIG. FIG. 3A is an example of the current position image included in the moving object information, and FIG. 3B is an example of incidental information regarding the current position image included in the moving object information.

In the example of FIG. 3, the current position image whose file name is "98765432.jpg" is taken at "12:12:12 on December 12, 2014", and the latitude and longitude of the shooting position are "+35.682343" respectively. , "+139.7735333", indicating that the shooting (advancing) direction is "30 degrees". Here, the latitude / longitude information is obtained by converting the coordinates in sexagesimal (degrees, minutes, seconds) into decimal numbers, and the shooting (traveling) direction is 0 degrees in the north direction, 90 degrees in the east direction, and 180 degrees in the south direction. , The direction when the west direction is 270 degrees is expressed by the angle.

In addition, the current location image in which the file name shown in FIG. 3A is "98765432.jpg" is the image of the current position of the pedestrian crossing in front of the moving body 105 when the own vehicle is going straight at an intersection with a traffic light at a green light. This is a photograph of a pedestrian nearby and a preceding vehicle approaching just before the pedestrian crossing.

Further, the transmission / reception unit 201 transmits the danger level information regarding the risk level determined by the risk level determination unit 204 (described later) to the mobile device 102 via the network 104. The method for determining the degree of risk will be described in detail later.

[1-2-3. Hazard Information Management Department]
The danger occurrence information management unit 202 includes the above-mentioned image database 103, and manages (stores) the danger occurrence information. That is, the danger occurrence information management unit 202 accumulates the danger point image which is an image of the surrounding situation of the moving body 105 that has encountered the danger event, and the shooting date and time, the shooting position, the shooting (progress) direction, etc. of the danger point image, etc. It is managed by associating it with the accompanying information of.

Here, an example of the danger occurrence information managed by the danger occurrence information management unit 202 will be described with reference to FIG. 4. FIG. 4A is an example of a danger point image included in the danger occurrence information, and FIG. 4B is an example of incidental information regarding the danger point image included in the danger occurrence information.

In the example of FIG. 4, the danger point image having the file name “000000123.jpg” was taken at “8:30:00 on December 1, 2014”, and the latitude and longitude of the shooting positions were “+35.711283” respectively. , "+139.704802", indicating that the shooting (advancing) direction is "30 degrees". Here, the latitude / longitude information is obtained by converting the coordinates in sexagesimal (degrees, minutes, seconds) into decimal numbers, and the shooting (traveling) direction is 0 degrees in the north direction, 90 degrees in the east direction, and 180 degrees in the south direction. , The direction when the west direction is 270 degrees is expressed by the angle.

Further, a dangerous event in each dangerous point image shown in FIG. 4A will be described. The dangerous event in the danger point image whose file name is "000000123.jpg" is a "hiyari hat of the encounter" in which the straight-ahead vehicle pops out from the left when the own vehicle, which is a moving body 105, goes straight on an intersection without a traffic light. .. The dangerous event in the danger point image whose file name is "00445566.jpg" is a pedestrian crossing the pedestrian crossing in front at a red light when the own vehicle, which is a moving body 105, goes straight at an intersection with a traffic light at a green light. The preceding vehicle approaching the pedestrian crossing suddenly brakes because of the pedestrian crossing. The dangerous event in the danger point image whose file name is "77088099.jpg" is that when the own vehicle, which is a moving body 105, turns right at an intersection with a traffic light at a green light, the right turn is completed before the oncoming vehicle approaches the intersection. When I hurried to try, there was a pedestrian crossing on the pedestrian crossing at the right turn, which was a "hearing hat with an oncoming crossing".

[1-2-4. Image analysis unit]
The image analysis unit 203 analyzes the traffic condition at the current location at the time of shooting the current location image based on the current location image included in the moving object information received by the transmission / reception unit 201. Further, the image analysis unit 203 analyzes the traffic information at the danger point when a danger event occurs, based on the danger point image included in the danger occurrence information stored in the danger occurrence information management unit 202.

Here, the traffic condition is a road traffic situation in which the mobile body 105 on which the mobile body device 102 is placed is placed, and specifically, the situation regarding a place including the road shape, the traffic environment, and the own vehicle. It is an action, etc. More specifically, the situation regarding the place including the road shape is, for example, an intersection, a single road (straight line), a curve, a narrow road, or the like. The traffic environment is, for example, the color of a traffic light, the presence or absence of a sign, the presence or absence of pedestrians and oncoming vehicles, the presence or absence of a preceding vehicle, the distance between vehicles, and the like. The own vehicle behavior is, for example, going straight, turning left or right, stopping, reversing, running speed, acceleration, and the like.

Next, an example of a method of analyzing the traffic condition based on the current position image will be described with reference to FIG. As shown in FIG. 3A, when the transmission / reception unit 201 receives the moving object information including the current position image whose file name is, for example, "98765432.jpg", the image analysis unit 203 i) changes the road shape. It is an intersection with a traffic light for the situation including the place, ii) There is a green light for the traffic environment, there are pedestrians near the pedestrian crossing in front, and the preceding vehicle is in front of the pedestrian crossing. Analyze the fact that you are approaching, iii) you are going straight for your own vehicle behavior.

Here, an example of a method of analyzing the traffic condition based on the current position image will be described more specifically. The situation regarding the place including the road shape can be detected, for example, by recognizing the lane markings and road markings drawn on the road surface from the current position image. The lane marking is, for example, a lane boundary line for indicating a traveling lane of a vehicle. Road markings are, for example, stop lines, "stop" marks, pedestrian crossings, diamond marks indicating the presence of pedestrian crossings, straight-ahead and left-right turning arrows, and broken lines to indicate prohibition of lane changes. For example, when these road markings are present, it is possible to detect that the road is an intersection. Further, for example, when these road markings do not exist and the lane marking is straight (or curved), it is possible to detect that the road is a single road (or a curve). Further, it is possible to detect whether or not the road is a narrow road, for example, by the distance between adjacent lane markings.

Further, the traffic environment can be detected by recognizing an image from a current position image by using color characteristics and shape characteristics of an object such as a signal, a sign, a pedestrian, or another vehicle. In addition, for example, by recognizing the headlight and the taillight as images, it is possible to detect whether the other vehicle is an oncoming vehicle or a preceding vehicle. In addition, for example, by recognizing the license plate of the preceding vehicle and the characters on the license plate as an image, the inter-vehicle distance can be detected based on the size of the license plate in the image and the known shooting magnification.

Further, the behavior of the own vehicle can be detected by paying attention to the movement of, for example, a road and surrounding buildings in a plurality of images of current locations that are close to each other in time. Alternatively, it may be detected by a change in the shooting position of two current position images that are close to each other in time.

[1-2-5. Danger level judgment unit]
The danger level determination unit 204 is a moving body 105 at the time of shooting the current position image based on the similarity between the current position image received by the transmission / reception unit 201 and the danger point image stored in the danger occurrence information management unit 202. Determine the risk of the situation in which. More specifically, the risk determination unit 204 determines the traffic conditions at the current position at the time of taking the current position image analyzed by the image analysis unit 203 and the traffic conditions at the dangerous point when a dangerous event occurs. The risk level described above is determined based on the similarity. Further, the risk level determination unit 204 generates risk level information regarding the determined risk level.

Next, an example of a method for determining the degree of danger will be described with reference to FIGS. 3 and 4. For example, the danger occurrence information management unit 202 manages the danger point image and its accompanying information shown in FIGS. 4A and 4B, respectively, and the transmission / reception unit 201 manages the file name shown in FIG. A case where the current position image of ".jpg" is received will be described. In this case, the risk determination unit 204 is, based on the traffic conditions analyzed by the image analysis unit 203, i) an intersection with a signal regarding the situation regarding the place including the road shape, and ii) a green light regarding the traffic environment. Danger Occurrence Information Management Department based on the fact that there are pedestrians near the pedestrian crossing in front, that the preceding vehicle is approaching just before the pedestrian crossing, and that iii) the vehicle is going straight ahead. The similarity (degree of similarity) between the danger point image and the current point image whose file name managed by 202 is "00445566.jpg" is calculated. When the calculated similarity is, for example, "0.8", the risk determination unit 204 is in a situation where the moving body 105 is placed at the time of shooting the current position image whose file name is "98765432.jpg". It is determined that the degree of danger of is "0.8". The degree of danger is, for example, a score from "0" to "1" (particle size in increments of 0.1), "0" means that there is no degree of danger, and "1" means that the degree of danger is the highest. Is shown.

Here, an example of a method of calculating the similarity between the danger point image and the current position image will be described. For example, the similarity can be calculated by using an image retrieval method called CBIR (Content-Based Image Retrieval). This CBIR is known as a method of detecting an image similar to a given image from a database based on the similarity of image features such as color, object shape and texture (texture and pattern on the surface of the object). .. According to this CBIR, it is possible to obtain a highly similar danger point image by searching the danger occurrence information management unit 202 based on the similarity of the image feature amount for the traffic condition analyzed by the image analysis unit 203. Become.

In addition, an example of a method of generating risk information regarding the determined risk level will be described. For example, the determined risk level (“0.8” in the above-mentioned example of the risk level determination method) can be adopted as the risk level information as it is, but the risk level information is used by the user of the moving body 105 (for example, the vehicle). Anything that can be output to call attention to the driver) may be used, and for example, arbitrary character information or voice may be used. It is not necessary to generate this risk information one by one every time the risk level is determined, and any character information or voice may be selected as the risk level information from the character information or voice registered in advance. .. Here, for example, when arbitrary character information, voice, or the like is adopted as the risk level information, it is desirable to adopt character information, voice, or the like that allows the user of the mobile body 105 to understand the risk level.

[1-2-6. Control unit]
The control unit 205 realizes the function of the server device 101 by managing and controlling the transmission / reception unit 201, the danger occurrence information management unit 202, the image analysis unit 203, and the risk level determination unit 204 described above.

[1-3. Configuration of mobile device]
[1-3-1. Overall configuration of mobile device]
Next, the overall configuration of the mobile device 102 will be described in detail with reference to FIG. FIG. 5 is a block diagram showing a configuration of the mobile device 102 according to the first embodiment.

As shown in FIG. 5, the mobile device 102 includes a transmission / reception unit 501 (an example of a second transmission unit and a second reception unit), an input reception unit 502, an output unit 503, and a control unit 504.

The mobile device 102 includes a microprocessor, RAM, ROM, a hard disk, and the like, which are not specifically shown. Computer programs are stored in these RAMs, ROMs, and hard disks. When the microprocessor operates according to this computer program, the mobile device 102 fulfills its function.

Each functional block such as the transmission / reception unit 501, the input reception unit 502, the output unit 503, and the control unit 504 is typically realized as an LSI which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include one or more functional blocks or a part of each functional block. Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI or ultra LSI depending on the degree of integration. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. After manufacturing the LSI, an FPGA that can be programmed or a reconfigurable processor that can reconfigure the connection and setting of the circuit cells inside the LSI may be used. Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. The application of biotechnology may be possible. Finally, each functional block of the mobile device 102 may be realized by software or may be realized by a combination of LSI and software. Also, the software may be tamper resistant.

[1-3-2. Transmitter / receiver]
The transmission / reception unit 501 transmits the mobile information received by the input reception unit 502 to the server device 101 via the network 104.

Further, the transmission / reception unit 501 receives the risk information transmitted from the server device 101 via the network 104.

[1-3-3. Input reception section]
The input receiving unit 502 receives the moving body information as an input. More specifically, the input receiving unit 502 receives an image of the current position, which is an image of the surrounding situation of the moving body 105 at the current position, and incidental information about the image of the current position as inputs.

The current location image is, for example, an image taken by an in-vehicle camera, a drive recorder, a smartphone camera, or the like. As for i) a method of shooting an image with various cameras, ii) a method of acquiring a shooting date and time, a shooting position, a shooting (traveling) direction, etc., any method generally disclosed may be used, and the description thereof will be described here. Omit.

[1-3-4. Output section]
The output unit 503 outputs attention-calling information for calling attention to the user of the mobile body 105 (or the mobile body device 102) based on the danger level information received by the transmission / reception unit 501. For example, when the alert information is character information, the character information for alerting the user by outputting the alert information to a display or the like (not shown) mounted on the moving body 105 (not shown). (For example, a character string stating "The risk of a rear-end collision is 0.8") is displayed on the display. Further, for example, when the alert information is voice, the alert information is output to a speaker or the like (not shown) mounted on the moving body 105 to alert the user. (For example, a voice saying "The risk of a rear-end collision is 0.8") is output from the speaker.

[1-3-5. Control unit]
The control unit 504 realizes the function of the mobile device 102 by managing and controlling the transmission / reception unit 501, the input reception unit 502, and the output unit 503 described above.

[1-4. Operation of danger judgment system]
Next, the operation (danger determination method) of the danger determination system 10 will be described with reference to FIG. FIG. 6 is a sequence diagram showing an operation flow of the danger determination system 10 according to the first embodiment.

First, the mobile device 102 receives the current position image obtained by photographing the surrounding situation of the mobile body 105 on which the mobile device 102 is mounted and the accompanying information regarding the current position image as inputs (S601). .. After that, the mobile device 102 transmits the received current position image and accompanying information to the server device 101 via the transmission / reception unit 501 (S602).

The server device 101 receives the current position image and accompanying information from the mobile device 102 via the transmission / reception unit 201 (S603). After that, the server device 101 analyzes the traffic condition at the current location at the time of shooting the current location image based on the received current location image and accompanying information by the image analysis unit 203 (S604). After that, the server device 101 receives the current position image from the similarity between the danger point image managed by the danger occurrence information management unit 202 and the current position image based on the traffic condition analyzed by the danger level determination unit 204. The risk level of the situation where the moving body 105 is placed at the time of shooting is determined, and the risk level information regarding the determined risk level is generated (S605).

At this time, the danger point image corresponding to the current position (for example, the intersection A represented by the current position image whose file name is "98765432.jpg" shown in (a) of FIG. 3) is managed by the danger occurrence information management unit 202. The dangerous point image corresponding to a point other than the current point (for example, the intersection B shown in the dangerous point image whose file name shown in FIG. 4A is "00445566.jpg") is dangerous. When managed by the occurrence information management unit 202, the risk level determination unit 204 is based on the similarity between the danger point image corresponding to a point other than the current point (for example, the above intersection B) and the current position image. Determine the degree of risk.

Further, when the danger point image corresponding to the current point (for example, the intersection A) and the danger point image corresponding to the point other than the current point (for example, the intersection B) are managed by the danger occurrence information management unit 202. Searches for a danger point image having the highest similarity to the current position image among the danger point image corresponding to the current point and the danger point image corresponding to a point other than the current point. As a result, when the similarity between the danger point image corresponding to the point other than the current position (for example, the intersection B) and the current position image is the highest, the risk determination unit 204 corresponds to the point other than the current position. The degree of danger is determined based on the similarity between the image of the danger point and the image of the current location. When the similarity between the danger point image corresponding to the current position and the current location image is the highest, the risk determination unit 204 is based on the similarity between the danger point image corresponding to the current location and the current location image. The degree of danger may be determined.

After that, when the risk level is within a predetermined range, the server device 101 transmits the generated risk level information to the mobile device 102 via the transmission / reception unit 201 (S606). The range within a predetermined range is, for example, a case where the degree of danger exceeds a preset threshold value (for example, “0.5”). Further, the mobile device 102 that is the transmission destination of the danger level information is, for example, the mobile device 102 that is the transmission source of the mobile information received by the server device 101.

The mobile device 102 receives the risk information regarding the risk determined by the server device 101 from the server device 101 via the transmission / reception unit 501 (S607). After that, the mobile device 102 outputs the caution information for calling the user of the mobile 105 (or the mobile device 102) based on the received risk information on the output unit 503. (S608).

[1-5. Application example of danger judgment system]
Next, an example of a service to which the danger determination system 10 according to the first embodiment is applied will be described with reference to FIG. 7. FIG. 7 is a diagram for explaining an example of a service to which the danger determination system 10 according to the first embodiment is applied.

The group 700 shown in FIG. 7A is, for example, a company, a group, a household, or the like, and the scale thereof does not matter. There are a plurality of mobile devices 102 and communication devices 703 in the group 700. Each of the plurality of mobile devices 102 is a device that can be directly connected to the Internet (for example, a car navigation system with a communication module, a smartphone, a tablet, a personal computer, etc.), or a device that cannot be directly connected to the Internet by itself (for example,). , Car navigation without communication module, etc.). If each of the plurality of mobile devices 102 is a device that cannot be directly connected to the Internet by itself, it may be a device that can be connected to the Internet via the communication device 703. Further, in the group 700, there is a user 701 who uses a plurality of mobile devices 102.

The above-mentioned server device 101 exists in the data center operating company 710 shown in FIG. 7A. The server device 101 is a virtualization server that cooperates with various devices via the Internet, and mainly manages huge data (big data) and the like that are difficult to handle with a normal database management tool or the like. The data center operating company 710 manages data, manages the server device 101, and operates a data center that manages the data center. The services performed by the data center operating company 710 will be described in detail later.

Here, the data center operating company 710 is not limited to a company that only manages data and operates the server device 101. As shown in FIG. 7 (b), for example, an apparatus maker who develops and manufactures one of a plurality of mobile apparatus 102 also manages data, server apparatus 101, and the like. If so, the equipment manufacturer corresponds to the data center operating company 710.

Further, the data center operating company 710 is not limited to one company. As shown in FIG. 7 (c), for example, when the device manufacturer and another management company jointly or share data management and operation of the server device 101, either or both of them perform data. It corresponds to the center operating company 710.

The service provider 720 shown in FIG. 7A has a server device 721. The scale of the server device 721 may be limited, for example, a memory in a personal personal computer or the like. It is possible that the service provider 720 does not have the server device 721.

Next, the flow of information in the service to which the danger determination system 10 is applied will be described with reference to FIG. 7A.

First, the mobile device 102 of the group 700 operates a data center by capturing an image of the current position of the mobile body 105 (see FIG. 1) on which the mobile device 102 is mounted and incidental information related to the current position image. It is transmitted to the server device 101 of the company 710. As a result, the server device 101 collects the current position image and the accompanying information transmitted from the mobile device 102 ((A) shown in FIG. 7). The current location image and accompanying information may be provided directly to the server device 101 from the plurality of mobile devices 102 themselves via the Internet. Further, the current position image and the accompanying information may be provided from the plurality of mobile devices 102 to the server device 101 via the communication device 703.

Next, the server device 101 of the data center operating company 710 provides the accumulated current position image and accompanying information to the service provider 720 in a fixed unit. Here, the unit of information provided by the data center operating company 710 may be a unit capable of organizing the accumulated current position image and accompanying information and providing the service provider 720, or the service provider 720 requests. It may be a unit that has been used. Further, the unit of information provided by the data center operating company 710 does not have to be constant, and the amount of information provided may change depending on the situation.

The current location image and accompanying information are stored in the server device 721 owned by the service provider 720 as needed ((B) shown in FIG. 7). After that, the service provider 720 organizes the current location image and the accompanying information into information suitable for the service provided to the user (risk degree information or alert information output based on the risk information) and provides the user with the information. The user to be provided may be a user 701 who uses a plurality of mobile devices 102, or an external user 702.

The current location image and accompanying information may be provided directly from the service provider 720 to the user 702 (or the user 701) ((E) or (F) shown in FIG. 7), or the data center operation. It may be provided to the user 701 via the server device 101 of the company 710 again ((C) and (D) shown in FIGS. 7). Further, the server device 101 of the data center operating company 710 may organize the current location image and accompanying information into information suitable for the service provided to the user and provide the service provider 720. The user 701 and the user 702 may be different users or the same user.

[1-6. effect]
Next, the effect obtained by the danger determination system 10 according to the first embodiment will be described. As described above, the danger level determination unit 204 is a case where the danger occurrence information corresponding to the current position (for example, the above intersection A) indicated by the accompanying information regarding the current position image is not stored in the danger occurrence information management unit 202. Also, the degree of danger is determined based on the similarity between the danger point image included in the danger occurrence information corresponding to the point other than the current point (for example, the intersection B) and the current position image. As a result, even when the moving body 105 passes through a point other than the danger point corresponding to the danger occurrence information managed by the danger occurrence information management unit 202 (for example, the above intersection A), the danger level is accurately determined. For example, it is possible to notify or warn the user of the moving body 105 of the danger.

[1-7. Modification of Embodiment 1]
[1-7-1. Modification 1]
The danger point image managed by the server device 101 in the danger occurrence information management unit 202 does not have to be the captured image itself, and is processed into an image in which the danger degree determination unit 204 can easily calculate the similarity with the current position image. Alternatively, one or more danger point images processed in this way may be newly associated with the original danger point image and managed.

Specifically, for example, the image features related to the situation including the road shape, the traffic environment, and the behavior of the own vehicle may be processed into a binarized image that highlights the image features, or the binarization thereof. The image may be managed by associating it with the original danger point image.

Furthermore, even if the image feature amount related to the traffic situation is managed in association with the original danger point image as a plurality of binarized images according to the situation regarding the place including the road shape, the traffic environment, the behavior of the own vehicle, etc. good. In this case, the risk determination unit 204 calculates not the similarity between the original danger point image and the current location image, but the similarity between the danger point image processed into a binarized image and the current location image, for example. Is desirable.

[1-7-2. Modification 2]
The accompanying information of the danger point image managed by the server device 101 in the danger occurrence information management unit 202 does not have to be the value itself acquired at the time of shooting, and the similarity between the current position image and the danger point image in the danger level determination unit 204. It may be replaced with a particle size that is meaningful in calculating (or it is easy to calculate the similarity), or ancillary information of the particle size may be newly added.

Specifically, the particle size that is meaningful (or easy to calculate the similarity) for the shooting date and time is, for example, 24 time zones, morning / afternoon, month / day, day of the week, season, and holidays every hour from 0:00. , So-called Goto days (5th, 10th, 15th, 20th, 25th, 30th) and the like. In addition, the particle size that is meaningful (or easy to calculate the similarity) for the shooting position is, for example, an area where map information is divided into rectangular shapes of a uniform size, or a road section where a road is divided into predetermined distances. Is. Furthermore, the particle size that is meaningful (or easy to calculate the similarity) in the shooting (traveling) direction is, for example, four directions of north, south, east, and west, eight directions including northeast / southeast / northwest / southwest, or up / of the road. Downhill, etc.

Further, the accompanying information of the danger point image managed by the server device 101 in the danger occurrence information management unit 202 does not have to be limited to the traffic condition at the time of shooting, and the mobile device 102 that has acquired the current position image is mounted. It may include various attribute information about the moving body 105. In this case, it is desirable that the risk level determination unit 204 calculate not only the similarity between the danger point image and the current position image but also the similarity including these various attribute information. Here, the various attribute information is, for example, the weight, displacement, vehicle type, etc. of the own vehicle which is the moving body 105. The vehicle type is, for example, a sedan, a wagon, a truck, or the like. These attribute information are fixed for each moving body 105, and can be acquired by, for example, setting in advance in the moving body device 102 according to the moving body 105.

[1-7-3. Modification 3]
The incidental information of the danger point image managed by the server device 101 by the danger occurrence information management unit 202 may include information explaining the content of the danger event such as an accident or a hiyari hat reflected in the danger point image. Further, the risk information generated by the server device 101 by the risk determination unit 204 may include a part or all of the information explaining the content of the danger event.

Here, an example of information for explaining the content of the dangerous event will be described with reference to FIG. 4A. As shown in FIG. 4A, the information for explaining the content of the dangerous event for the dangerous point image whose file name is “000000123.jpg” is, for example, “Hiyari hat of encounter”. Further, the information for explaining the content of the dangerous event for the dangerous point image whose file name is "00445566.jpg" is, for example, "rear-end collision hiyari hat". Further, the information for explaining the content of the dangerous event for the dangerous point image whose file name is "77088099.jpg" is, for example, "Hiyari hat with an oncoming crossing person".

An example of a method for generating risk information in this case will be described. For example, the server device 101 receives the current position image in which the file name shown in FIG. 3 (a) is "98765432.jpg" by the transmission / reception unit 201, and further, the risk level determination unit 204 shows it in FIG. 4 (a). When it is determined that the risk level is "0.8" based on the similarity between the danger point image whose file name is "00445566.jpg" and the current position image, it is generated by the risk level determination unit 204. The risk information is, for example, "the risk of a collision hire hat is 0.8".

[1-7-4. Modification 4]
The accompanying information of the current position image received by the server device 101 from the mobile device 102 does not have to be limited to the information related to the shooting of the current position image, and is a movement acquired from sensors separately mounted on the mobile body 105. Various sensor information (an example of the third sensor information) indicating the traveling state of the body 105 may be included. In this case, it is desirable that the risk level determination unit 204 calculate not only the similarity between the danger point image and the current position image but also the similarity including these various sensor information.

Here, the various sensor information is, for example, the traveling speed, acceleration, and angular velocity of the moving body 105 acquired from a position sensor such as GPS (Global Positioning System), an acceleration sensor, and an angular velocity sensor such as a gyro. Further, the sensor information is not limited to the above, and may include, for example, arbitrary information necessary for analyzing the traffic condition by the image analysis unit 203. For example, when the moving body 105 is a vehicle, the sensor information may include a steering wheel operation, a brake operation, an accelerator operation, a wiper operation, and the like, and these may be included in the vehicle such as CAN (Control Area Network). It can be obtained from the network.

Further, the accompanying information of the current position image received by the server device 101 from the mobile device 102 may include various attribute information regarding the mobile body 105 on which the mobile device 102 that has acquired the current position image is mounted. In this case, it is desirable that the risk level determination unit 204 calculate not only the similarity between the danger point image and the current position image but also the similarity including these various attribute information. Here, the various attribute information is, for example, the weight, displacement, vehicle type, etc. of the own vehicle which is the moving body 105. The vehicle type is, for example, a sedan, a wagon, a truck, or the like. These attribute information are fixed for each moving body 105, and can be acquired by, for example, setting in advance in the moving body device 102 according to the moving body 105.

[1-7-5. Modification 5]
The server device 101 does not necessarily have to individually and sequentially perform the traffic condition analysis processing in the image analysis unit 203 and the similarity calculation processing between the danger point image and the current position image in the danger level determination unit 204. For example, by using a machine learning method called deep learning, the traffic condition analysis process and the similarity calculation process may be performed in parallel.

[1-7-6. Modification 6]
In the danger occurrence information management unit 202, the server device 101 may also manage the severity of the danger event encountered by the moving body 105 in association with the incidental information of each danger point image. In this case, in the risk determination unit 204, for example, by multiplying the similarity between the danger point image having the highest similarity and the current position image and the severity managed in association with the danger point image, for example. , The degree of danger of the situation where the moving body 105 is placed at the time of taking the image of the current position may be determined.

Here, the severity may be anything as long as the severity of the dangerous event encountered by the mobile body 105 is expressed based on a predetermined standard. For example, the severity is the degree of seriousness such as whether or not an accident has occurred, the magnitude of human damage, or the magnitude of the amount of insurance paid, for example, a score from "0" to "1" (0. It is converted to 1-step particle size) and the like. The degree of seriousness may be obtained by any method from, for example, accident record information such as accidents or hiyari hats held by police or transportation companies, or accident assessment information held by non-life insurance companies. The description thereof is omitted here.

[1-7-7. Modification 7]
The server device 101 stores the current position image received by the transmission / reception unit 201 as a new danger point image in the danger occurrence information management unit 202 when the danger level determined by the danger level determination unit 204 is within a predetermined range. It may be (added) and managed by associating it with the accompanying information of the current location image. Here, the predetermined range is, for example, a case where a preset threshold value (for example, “0.5”) is exceeded.

[1-7-8. Modification 8]
The mobile device 102 that is the destination of the risk information regarding the risk determined by the server device 101 does not have to be the same as the mobile device 102 that is the source of the current position image received by the server device 101. For example, when the moving body 105 is a vehicle, the risk information regarding the degree of danger corresponding to the specific point determined based on the current position image transmitted from the preceding vehicle is slightly delayed from the same specific point (for example, moving). It may be transmitted to the moving body device 102 mounted on the following vehicle (another moving body 105) determined to pass (within a predetermined time after receiving the body information). As a result, it is possible to call attention to the user (driver) of the following vehicle.

(Embodiment 2)
[2-1. Overall configuration of danger judgment system]
The overall configuration of the danger determination system 10A according to the second embodiment will be described with reference to FIG. FIG. 8 is a block diagram showing a configuration of the danger determination system 10A according to the second embodiment. In the present embodiment, the same components as those in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 8, the danger determination system 10A includes a server device 101A and a plurality of mobile devices 102. The mobile device 102 is the same as that described in the first embodiment.

The server device 101A includes an index database 106 (an example of an index management unit) in addition to the image database 103 described in the first embodiment. In the index database 106, the danger index (danger determination information) regarding the degree of danger of a dangerous event such as an accident or a hiyari hat that has occurred in the past includes the date and time of occurrence of the dangerous event, the position of occurrence, and the direction of occurrence (that is, the traveling direction of the moving body 105). ) Etc. are linked and managed (stored). Here, the risk index is obtained or calculated in advance by any method from, for example, accident record information such as accidents or hiyari hats held by police or transportation companies, or accident assessment information held by non-life insurance companies. However, the description thereof will be omitted here.

The server device 101A is in a situation where the mobile body 105 is placed at the time of taking the current position image based on the similarity between the incidental information received from the mobile device 102 and the danger index managed in the index database 106. Determine the first risk level, which is the risk level of. Further, the server device 101A transmits the first risk level information regarding the determined first risk level to the mobile device 102 via the network 104.

Further, the server device 101A analyzes the traffic condition at the time of shooting the current position image based on the current position image received from the mobile device 102 and the accompanying information thereof, and the dangerous point managed in the image database 103. From the similarity between the image and the current position image, the second risk level, which is the risk level of the situation in which the moving body 105 is placed at the time of shooting the current position image, is determined. Further, when the determined second risk level is within a predetermined range, the server device 101A stores the second risk level information regarding the determined second risk level in the index database 106 as a new risk index (as a new risk index). Add) and manage by associating with the accompanying information of the current location image. Here, the predetermined range is, for example, a case where a preset threshold value (for example, “0.5”) is exceeded.

[2-2. Server device configuration]
[2-2-1. Overall configuration of server device]
Next, the overall configuration of the server device 101A will be described in detail with reference to FIG. FIG. 9 is a block diagram showing a configuration of the server device 101A according to the second embodiment.

As shown in FIG. 9, the server device 101A includes a danger determination information management unit 206 in addition to the transmission / reception unit 201, the danger occurrence information management unit 202, the image analysis unit 203, the risk level determination unit 204A, and the control unit 205A. There is.

The server device 101A is specifically composed of a microprocessor, RAM, ROM, a hard disk, or the like (not shown). Computer programs are stored in these RAMs, ROMs, and hard disks. When the microprocessor operates according to this computer program, the server device 101A fulfills its function.

The functional blocks of the server device 101A, such as the transmission / reception unit 201, the danger occurrence information management unit 202, the image analysis unit 203, the risk level determination unit 204A, the control unit 205A, and the danger determination information management unit 206, are typically integrated. It is realized as an LSI that is a circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include one or more functional blocks or a part of each functional block. Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI or ultra LSI depending on the degree of integration. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. After manufacturing the LSI, a programmable FPGA or a reconfigurable processor that can reconfigure the connection and setting of the circuit cells inside the LSI may be used. Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. The application of biotechnology may be possible. Finally, each functional block may be realized by software or may be realized by a combination of LSI and software. Also, the software may be tamper resistant.

[2-2-2. Transmitter / receiver]
The transmission / reception unit 201 is the same as that described in the first embodiment, but transmits the first risk level information regarding the first risk level determined by the risk level determination unit 204A to the mobile device 102.

[2-2-3. Hazard Information Management Department]
The danger occurrence information management unit 202 is the same as that described in the first embodiment.

[2-2-4. Image analysis unit]
The image analysis unit 203 is the same as that described in the first embodiment.

[2-2-5. Danger level judgment unit]
The danger level determination unit 204A is based on the similarity between the incidental information about the current position image received by the transmission / reception unit 201 and the danger determination information managed by the danger determination information management unit 206, at the time of shooting the current position image. The first risk level, which is the risk level of the situation in which the moving body 105 is placed, is determined, and the first risk level information regarding the determined first risk level is generated. The method of generating the first risk information is the same as that described in the first embodiment.

Here, an example of the first method for determining the degree of risk will be described. The danger determination information management unit 206 manages, for example, the danger determination information shown in FIG. 10, which will be described later, and the transmission / reception unit 201, for example, the accompanying information of the current position image whose file name shown in FIG. Is received. In this case, for example, based on the fact that the time zones of both are in the same 12 o'clock range, the positions (latitude and longitude) of both are close to each other, and the direction of travel is close to each other, the risk determination unit 204A is instructed to "December 8, 2014. Based on the similarity between the danger determination information of the dangerous event that occurred at "12:00:20 on the day" and the accompanying information of the current location image, it is determined that the first danger level is "0.9".

Further, in the danger level determination unit 204A, the transmission / reception unit 201 is based on the similarity between the danger point image managed by the danger occurrence information management unit 202 and the current position image based on the traffic condition analyzed by the image analysis unit 203. A second degree of danger, which is the degree of danger of the situation in which the moving body 105 is placed at the time of taking the received current position image, is determined. Further, the risk level determination unit 204A generates a second risk level information regarding the determined second risk level. Here, the method of calculating the similarity between the danger point image and the current position image, the method of determining the second degree of danger, and the method of generating the second degree of danger information are as described in the first embodiment. The same is true.

[2-2-6. Danger Judgment Information Management Department]
The danger determination information management unit 206 includes the index database 106 described above, and manages (stores) the danger determination information. That is, the danger judgment information management unit 206 accumulates danger judgment information regarding the degree of danger of accidents or dangerous events such as hiyari hats that have occurred in the past, and associates them with incidental information such as the date and time of occurrence, the position of occurrence, and the direction of occurrence (progress). To manage.

Here, an example of the danger determination information managed by the danger determination information management unit 206 will be described with reference to FIG. 10. FIG. 10 is a diagram showing an example of danger determination information according to the second embodiment.

In the example of FIG. 10, "December 1, 2014 8:30:00", "December 5, 2014 12:45:30", "December 8, 2014 12:00:20" , The danger judgment information of dangerous events such as accidents or hiyari hats that occurred at "14:00:45 on December 10, 2014" is managed in association with the date and time of occurrence, the position of occurrence, and the direction of occurrence (progress). Has been done. For example, at "8:30:00 on December 1, 2014", the degree of danger is high while traveling at the points where the latitude and longitude are "+35.711283" and "+139.704802" in the direction of travel "30 degrees", respectively. It indicates that a dangerous event such as an accident or a hiyari hat that is "0.5" has occurred.

The latitude and longitude information is obtained by converting the coordinates in sexagesimal (degrees, minutes, seconds) into decimal numbers, and the shooting (traveling) direction is 0 degrees northward, 90 degrees eastward, 180 degrees southward, and westward. Is expressed as an angle when the degree is 270 degrees. The degree of danger is, for example, a score from "0" to "1" (particle size in increments of 0.1), "0" indicates that there is no degree of danger, and "1" indicates that the degree of danger is the highest. ing. The method of expressing the degree of danger is not limited to this, and for example, the upper limit value and the particle size (for example, only an integer value or increments of 0.1) indicating that the degree of danger is the highest can be arbitrarily determined and adopted.

Further, the danger determination information management unit 206 uses the second danger degree information regarding the second danger degree as a new danger determination when the second danger degree determined by the danger degree determination unit 204A is within a predetermined range. It is accumulated (added) as information and managed by associating it with the accompanying information of the current location image. Here, the predetermined range is, for example, a case where a preset threshold value (for example, “0.5”) is exceeded.

[2-2-7. Control unit]
The control unit 205A realizes the function of the server device 101A by managing and controlling the transmission / reception unit 201, the danger occurrence information management unit 202, the image analysis unit 203, the danger level determination unit 204A, and the danger determination information management unit 206 described above. do.

[2-3. Mobile device]
The mobile device 102 is basically the same as that described in the first embodiment. The transmission / reception unit 501 (see FIG. 5) of the mobile device 102 transmits the current position image and its accompanying information to the server device 101A, and receives the first risk information from the server device 101A. Further, the output unit 503 (see FIG. 5) of the mobile device 102 outputs the attention alert information based on the received first danger level information.

[2-4. Operation of danger judgment system]
Next, the operation (danger determination method) of the danger determination system 10A will be described with reference to FIG. FIG. 11 is a sequence diagram showing an operation flow of the danger determination system 10A according to the second embodiment.

The mobile device 102 receives, as input, an image of the current position obtained by photographing the surrounding situation of the mobile body 105 on which the mobile device 102 is mounted, and incidental information related to the image of the current position, in the input receiving unit 502 (see FIG. 5). (S1101). After that, the mobile device 102 transmits the received current position image and accompanying information to the server device 101A via the transmission / reception unit 501 (S1102).

The server device 101A receives the current position image and accompanying information from the mobile device 102 via the transmission / reception unit 201 (S1103). After that, when the danger determination information corresponding to the current position indicated by the accompanying information regarding the received current position image is managed by the danger determination information management unit 206, the server device 101A uses the risk determination unit 204A to display the danger determination information. The first risk level is determined based on the similarity with the accompanying information regarding the current position image, and the first risk level information regarding the determined first risk level is generated (S1104). After that, when the first risk level is within the predetermined range, the server device 101A transmits the generated first risk level information to the mobile device 102 via the transmission / reception unit 201 (S1105).

The mobile device 102 receives the first danger level information from the server device 101A via the transmission / reception unit 501 (S1106). After that, the mobile device 102 uses the output unit 503 to call attention to the user of the mobile 105 (or the mobile device 102) based on the first danger level information received. Is output (S1107).

Further, the server device 101A analyzes the traffic condition at the time of taking the current position image by the image analysis unit 203 based on the received current position image and the accompanying information (S1108). After that, the server device 101A receives the current position image based on the similarity between the danger point image managed by the danger occurrence information management unit 202 and the current position image based on the traffic condition analyzed by the risk determination unit 204A. The second danger level of the situation where the moving body 105 is placed at the time of shooting is determined, and the second danger level information regarding the determined second danger level is generated (S1109). After that, the server device 101A accumulates (adds) the determined second danger degree information as new danger determination information by the danger determination information management unit 206, and manages it in association with the accompanying information of the current position image (S1110). ).

[2-5. Application example of danger judgment system]
The application example of the danger determination system 10A according to the second embodiment is the same as that described in the first embodiment.

[2-6. effect]
Next, the effect obtained by the danger determination system 10A according to the second embodiment will be described. As described above, the risk level determination unit 204A determines the first risk level based on the similarity between the risk determination information and the accompanying information regarding the current position image. Since this first risk determination process can be executed in a shorter time than the similarity calculation process between the danger point image and the current position image, the mobile device 105 can quickly output the alert information. can do.

[2-7. Modification of Embodiment 2]
[2-7-1. Modification 1]
The danger point image managed by the server device 101A in the danger occurrence information management unit 202 does not have to be the captured image itself, and is processed into an image in which the danger degree determination unit 204A can easily calculate the similarity with the current position image. Alternatively, one or more danger point images processed in this way may be newly associated with the original danger point image and managed.

Specifically, for example, the image features related to the situation including the road shape, the traffic environment, and the behavior of the own vehicle may be processed into a binarized image that highlights the image features, or the binarized image thereof. May be managed by associating it with the original danger point image.

Furthermore, even if the image feature amount related to the traffic situation is managed in association with the original danger point image as a plurality of binarized images according to the situation regarding the place including the road shape, the traffic environment, the behavior of the own vehicle, etc. good. In this case, the risk determination unit 204A calculates not the similarity between the original danger point image and the current position image, but the similarity between the danger point image processed into a binarized image and the current position image, for example. Is desirable.

[2-7-2. Modification 2]
The accompanying information of the danger judgment information managed by the server device 101A by the danger judgment information management unit 206 does not have to be the value itself acquired in advance, and the danger degree judgment unit 204A calculates the similarity with the incidental information of the current position image. It may be replaced with a particle size that is meaningful (or it is easy to calculate the similarity), or additional information of the particle size may be added.

Specifically, the particle size that is meaningful (or easy to calculate the similarity) for the date and time of occurrence is, for example, 24 time zones every hour from 0:00, morning / afternoon, month, day, day of the week, season, holiday, etc. So-called Goto days (5th, 10th, 15th, 20th, 25th, 30th) and the like. In addition, the particle size that is meaningful (or easy to calculate the similarity) for the position of occurrence is, for example, an area where map information is divided into rectangular shapes of a uniform size, or a road section where a road is divided into predetermined distances. Is. Furthermore, the particle size that is meaningful (or easy to calculate the similarity) in the direction of occurrence (progress) is, for example, four directions of north, south, east, and west, eight directions including northeast / southeast / northwest / southwest, or up / of the road. Downhill, etc.

Further, the incidental information of the danger determination information managed by the server device 101A in the danger determination information management unit 206 does not have to be limited to the traffic condition at the time of occurrence, and various attributes related to the moving body 105 that encountered the danger event. Information may be included. The various attribute information is, for example, the weight, displacement, vehicle type, etc. of the own vehicle, which is the moving body 105. The vehicle type is, for example, a sedan, a wagon, a truck, or the like. These attribute information may be obtained by any method from, for example, accident record information such as accidents or hiyari hats held by police or transportation companies, or accident assessment information held by non-life insurance companies. The description thereof is omitted here.

[2-7-3. Modification 3]
The incidental information of the danger determination information managed by the server device 101A by the danger determination information management unit 206 may include information explaining the contents of a dangerous event such as an accident or a hiyari hat that has occurred in the past. Further, the first risk information generated by the server device 101A by the risk determination unit 204A may include a part or all of the information explaining the content of the danger event.

Here, an example of information for explaining the content of the dangerous event will be described with reference to FIG. 10. As shown in FIG. 10, the information explaining the content of the dangerous event that occurred at “8:30:00 on December 1, 2014” is, for example, “a hiyari hat of the encounter”. Further, the information explaining the content of the dangerous event that occurred at "12:45:30 on December 5, 2014" is, for example, "rear-end collision hiyari hat". Further, the information explaining the content of the dangerous event that occurred at "12:00:20 on December 8, 2014" is, for example, "a right-handed accident with an oncoming vehicle". Further, the information explaining the content of the dangerous event that occurred at "14:00:45 on December 10, 2014" is, for example, "Hiyari hat with an oncoming crossing person".

An example of the first method of generating risk information in this case will be described. For example, the server device 101A receives the accompanying information of the current position image whose file name is "98765432.jpg" in the transmission / reception unit 201, and further, the risk determination unit 204A "December 2014". If it is determined that the first risk level is "0.9" based on the similarity with the risk judgment information of the dangerous event that occurred at "12:00:20 on the 8th", the risk level judgment unit The first risk information generated by the 204A is, for example, "the risk of a right-handed accident with an oncoming vehicle is 0.9".

[2-7-4. Modification 4]
The server device 101A does not necessarily have to individually and sequentially perform the traffic condition analysis process in the image analysis unit 203 and the similarity calculation process between the danger point image and the current point image in the risk level determination unit 204A. For example, by using a machine learning method called deep learning, the traffic condition analysis process and the similarity calculation process may be performed in parallel.

[2-7-5. Modification 5]
When the second danger level determined by the danger level determination unit 204A is within a predetermined range, the server device 101A uses the danger occurrence information management unit 202 to display the current position image received by the transmission / reception unit 201 as a new danger point. It may be accumulated (added) as an image and managed in association with the accompanying information. Here, the predetermined range is, for example, a case where a preset threshold value (for example, “0.5”) is exceeded.

[2-7-6. Modification 6]
The server device 101A uses the risk determination information referred to when the risk determination unit 204A determines the first risk when the second risk determined by the risk determination unit 204A is within a predetermined range. It may be rewritten or deleted from the danger determination information management unit 206. Here, the predetermined range is, for example, a case where the threshold value is lower than a preset threshold value (for example, “0.5”).

[2-7-7. Modification 7]
The mobile device 102 that is the destination of the first risk information regarding the first risk determined by the server device 101A is the same as the mobile device 102 that is the source of the current position image received by the server device 101A. It doesn't have to be. For example, when the moving body 105 is a vehicle, the first risk information regarding the first risk of the specific point determined based on the current position image transmitted from the preceding vehicle is slightly delayed from the same specific point. It may be transmitted to the moving body device 102 mounted on the following vehicle passing by. As a result, it is possible to call attention to the user (driver) of the following vehicle.

(Other variants)
The risk determination method and the like according to one or more embodiments have been described above based on the above embodiments 1 and 2, but the present disclosure is not limited to these embodiments 1 and 2. As long as it does not deviate from the gist of the present disclosure, one or a form constructed by applying various modifications that can be conceived by those skilled in the art to the first and second embodiments, and a combination of components in different embodiments or modifications is also possible. It may be included in the range of a plurality of embodiments. For example, the above embodiments 1 and 2 may be combined, respectively.

Further, for example, in each of the above embodiments, the moving body 105 is used as a vehicle, and the vehicle includes an automobile, a two-wheeled vehicle, a train, a bicycle, and the like. Further, the moving body 105 is not limited to the vehicle, and may be, for example, a smartphone or a tablet.

A part or all of the components constituting each of the above devices may be composed of an IC card or a single module that can be attached to and detached from each device. The IC card or the module is a computer system composed of a microprocessor, ROM, RAM and the like. The IC card or the module may include the above-mentioned super multifunctional LSI. When the microprocessor operates according to a computer program, the IC card or the module achieves its function. This IC card or this module may have tamper resistance.

The present invention may be the method shown above. Further, it may be a computer program that realizes these methods by a computer, or it may be a digital signal composed of the computer program. Further, the present invention also relates to a recording medium capable of computer-readable reading of the computer program or the digital signal, such as a flexible disk, a hard disk, a CD-ROM, MO, DVD, a DVD-ROM, a DVD-RAM, or a BD (Blu-ray). It may be recorded in a trademark) Disc), a semiconductor memory, or the like. Further, it may be the digital signal recorded on these recording media. Further, the present invention may transmit the computer program or the digital signal via a telecommunication line, a wireless or wired communication line, a network typified by the Internet, data broadcasting, or the like. Further, the present invention is a computer system including a microprocessor and a memory, in which the memory stores the computer program, and the microprocessor may operate according to the computer program. It is also carried out by another independent computer system by recording and transferring the program or the digital signal on the recording medium, or by transferring the program or the digital signal via the network or the like. It may be.

The danger determination method according to the present invention is useful for a danger determination system or the like that manages the degree of danger corresponding to a point where a moving object is located.

10,10A Danger determination system 101, 101A, 721 Server device 102 Mobile device 103 Image database 104 Network 105 Mobile 106 Index database 201,501 Transmission / reception unit 202 Danger occurrence information management unit 203 Image analysis unit 204, 204A Danger level determination unit 205, 205A, 504 Control unit 206 Danger judgment information management unit 502 Input reception unit 503 Output unit 700 Group 701,702 User 703 Communication device 710 Data center operating company 720 Service provider

Claims (11)

It is a judgment method executed by a computer.
The computer corresponds to the danger point information indicating the danger point where the danger event has occurred and the first image obtained by extracting the image feature amount related to the traffic situation from the image taken of the occurrence situation of the danger event at the danger point. Equipped with a hazard information management unit that stores one or more attached hazard information
The determination method is
A step of acquiring the moving body information in which the current position information indicating the current position of the moving body and the second image obtained by capturing the surrounding situation of the moving body at the current position are associated with each other.
The danger occurrence information corresponding to the current location indicated by the current location information included in the moving body information is not stored in the danger occurrence information management unit, and the danger corresponding to a point other than the current location. When the occurrence information is stored in the danger occurrence information management unit, the second image included in the moving body information and one or more of the danger occurrence information stored in the danger occurrence information management unit. Including a step of determining the degree of risk corresponding to the current location based on the similarity with the first image included.
The danger occurrence information further includes the severity of the danger event associated with the danger point information and the first image.
The severity is the presence or absence of an accident or the degree of damage caused by the accident.
In the determination step, the second image included in the moving body information is similar to the first image included in one or more danger occurrence information stored in the danger occurrence information management unit. A determination method for determining the degree of danger corresponding to the current position by obtaining the sex and executing an operation using the similarity and the severity associated with the first image. In the determination step, the danger occurrence information corresponding to the current location indicated by the current location information included in the moving body information and the danger occurrence information corresponding to a point other than the current location are the danger occurrence information. When stored in the management unit, the second image included in the moving body information and the first image included in the danger occurrence information corresponding to a point other than the current point.
The determination method according to claim 1, wherein the degree of danger corresponding to the current position is determined based on the similarity with the image of. The determination method further comprises
Based on the first image, the first traffic condition at the dangerous point when the dangerous event occurs is analyzed, and based on the second image, the second traffic condition at the current point is analyzed. Includes steps to analyze
The determination method according to claim 1, wherein in the determination step, the degree of danger corresponding to the current location is determined based on the similarity between the analyzed first traffic condition and the second traffic condition. The determination method further comprises
When the degree of danger corresponding to the determined current point is within a predetermined range, the current point information included in the moving body information is used as the dangerous point information, and the second item included in the moving body information. The step of generating the danger occurrence information using the image of 2 as the first image, and
The determination method according to any one of claims 1 to 3, comprising a step of adding the generated danger occurrence information to the danger occurrence information management unit. In the acquisition step, the mobile information transmitted from the mobile device mounted on the mobile is received via the network.
The determination method further comprises
Claims 1 to 1, which include a step of transmitting risk information regarding the risk to the mobile device that has transmitted the mobile information when the risk corresponding to the determined current position is within a predetermined range. The determination method according to any one of 4. In the acquisition step, the mobile information transmitted from the mobile device mounted on the mobile is received via the network.
The determination method further comprises
When the degree of danger corresponding to the determined current position is within a predetermined range, another moving body is included in the moving body information within a predetermined time after receiving the moving body information. Any of claims 1 to 4, which includes a step of transmitting risk information regarding the risk to a mobile device mounted on the other mobile when it is determined that the current location is to be passed as indicated by the current location information. The determination method according to item 1. The determination method further comprises
Including a step of storing the danger determination information in which the danger level corresponding to the determined current location and the current location information are associated with each other in the danger determination information management unit.
In the determination step, when the danger determination information corresponding to the current location indicated by the current location information included in the moving object information is stored in the danger determination information management unit, it is included in the danger determination information. The determination method according to any one of claims 1 to 6, wherein the risk level is used. The moving body information further includes sensor information indicating a traveling state of the moving body, which is associated with the current position information and the second image.
In the determination step, the second image included in the moving body information is similar to the first image included in one or more of the danger occurrence information stored in the danger occurrence information management unit. The determination method according to any one of claims 1 to 7, wherein the property is determined, and the degree of danger corresponding to the current position is determined based on the similarity and the sensor information. Danger in which the danger point information indicating the danger point where the danger event has occurred is associated with the first image obtained by extracting the image feature amount related to the traffic situation from the image taken of the occurrence situation of the danger event at the danger point. The danger occurrence information management department that stores one or more occurrence information,
A receiving unit that receives the moving body information in which the current position information indicating the current position of the moving body and the second image obtained by capturing the surrounding situation of the moving body at the current position are associated with each other.
The danger occurrence information corresponding to the current location indicated by the current location information included in the moving body information is not stored in the danger occurrence information management unit, and the danger corresponding to a point other than the current location. When the occurrence information is stored in the danger occurrence information management unit, the second image included in the moving body information and one or more of the danger occurrence information stored in the danger occurrence information management unit. A determination unit for determining the degree of danger corresponding to the current position based on the similarity with the included first image is provided.
The danger occurrence information further includes the severity of the danger event associated with the danger point information and the first image.
The severity is the presence or absence of an accident or the degree of damage caused by the accident.
The determination unit has similarities between the second image included in the moving body information and the first image included in one or more of the danger occurrence information stored in the danger occurrence information management unit. A computer that determines the degree of danger corresponding to the current location by performing an operation using the similarity and the degree of severity associated with the first image. It is a system equipped with a determination device and an output device mounted on a moving body, and manages the degree of danger corresponding to the point where the moving body is located.
The determination device is
Danger in which the danger point information indicating the danger point where the danger event has occurred is associated with the first image obtained by extracting the image feature amount related to the traffic situation from the image taken of the occurrence situation of the danger event at the danger point. The danger occurrence information management department that stores one or more occurrence information,
A first receiving unit that receives the moving body information in which the current position information indicating the current position of the moving body and the second image obtained by capturing the surrounding situation of the moving body at the current point are associated with each other.
The danger occurrence information corresponding to the current location indicated by the current location information included in the moving body information is not stored in the danger occurrence information management unit, and the danger corresponding to a point other than the current location. When the occurrence information is stored in the danger occurrence information management unit, the second image included in the moving body information and one or more of the danger occurrence information stored in the danger occurrence information management unit. A determination unit that determines the degree of danger corresponding to the current location based on the similarity with the first image included.
A first transmission unit for transmitting risk information regarding the risk to the output device is provided.
The output device is
A second transmission unit that transmits the mobile body information to the determination device, and
A second receiving unit that receives the risk information transmitted from the determination device, and
It is provided with an output unit that outputs attention-calling information for calling attention to the user of the moving object based on the received danger level information.
The danger occurrence information further includes the severity of the danger event associated with the danger point information and the first image.
The severity is the presence or absence of an accident or the degree of damage caused by the accident.
The determination unit has similarities between the second image included in the moving body information and the first image included in one or more of the danger occurrence information stored in the danger occurrence information management unit. A system for determining the degree of danger corresponding to the current position by performing an operation using the similarity and the degree of severity associated with the first image. An output device used in the system according to claim 10.
A second transmission unit that transmits to the determination device the moving body information in which the current position information indicating the current position of the moving body and the second image showing the surrounding situation of the moving body at the current position are associated with each other.
A second receiving unit that receives the risk information transmitted from the determination device, and
An output device including an output unit that outputs attention-calling information for calling attention to the user of the moving object based on the received danger level information.

Images