JP2020109675A - Information collecting device, information collecting server, and information collecting system - Google Patents

Abstract

To provide an information collecting device, an information collecting server, and an information collecting system capable of collecting information about an event that occurs at an unspecified place on a route such as a road on which a moving object moves and providing the information to the moving object.SOLUTION: In an information collecting system 100, an information collecting device (terminal device 10) comprises: a passenger state information collecting unit that collects a passenger state of a moving object as passenger state information; a state change detection unit that detects a state change of the passenger based on the passenger state information and generates state change detection information; an external photographing unit for photographing an external area of the moving body to generate a photographed image; and an external situation information transmission unit that generates external situation information including the photographed image according to the state change detection information and transmits the external situation information via a communication path.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information collecting device, an information collecting server, and an information collecting system.

BACKGROUND ART Conventionally, a traffic information providing system that collects information from cameras installed at an intersection and transmits the situation near the intersection to a vehicle near the intersection has been studied (Patent Document 1).

JP, 2011-258068, A

The traffic information providing system as disclosed in Patent Document 1 can only obtain information on a specific place on a road such as an intersection where a camera is installed. Therefore, for example, even if an event occurs in a place other than the specific place, it is not possible to collect the information and provide the information based on the collected information.

The present invention has been made in view of the above points, and collects information on an event that occurs at an unspecified location of a route such as a road on which a mobile body moves, and provides the information to the mobile body. It is an object to provide an information collecting device, an information collecting server, and an information collecting system that enable the information collecting device.

The invention according to claim 1 of the present application is an information collecting device, and based on the passenger status information, a passenger status information collecting unit that collects a status of a passenger of a moving body as passenger status information. A state change detection unit that detects a state change of an occupant and generates state change detection information, an external photographing unit that photographs the outside of the moving body and generates a captured image, and the state change detection information according to the state change detection information. An external situation information transmission unit that generates external situation information including a captured image and transmits the external situation information via a communication path.

The invention according to claim 7 of the present application is an information collecting system, which is provided in a moving body, and which collects a passenger's state of the moving body as passenger state information, A state change detection unit that detects a state change of the occupant and generates state change detection information based on the occupant state information; and a captured image generated by photographing the outside of the moving body provided in the moving body. And an external situation information generation unit that generates external situation information including the captured image according to the state change detection information.

The invention according to claim 10 of the present application is an information collecting server, which includes an external captured image of the moving body generated in response to detection of a state change of an occupant riding on the moving body. An external situation information receiving unit that receives situation information via a communication path, and a determination unit that analyzes the captured image included in the external situation information and determines the usefulness of the external situation information are useful. An external situation information distribution unit that distributes the determined external situation information via a communication path.

1 is an overall configuration diagram of an information collecting system of the present invention. It is a figure which shows the example of attachment of a terminal device. It is a functional block diagram of a terminal device and a server. It is an example of an operation routine of the passenger monitoring unit. It is an example of an operation routine of the image extraction unit. It is an example of an operation routine of the own position specifying unit. It is an example of an operation routine of the content selection unit. It is an example of an operation routine of the information receiving unit. It is an example of an operation routine of the content selection unit. It is a functional block diagram of a terminal device and a server.

Hereinafter, preferred embodiments of the present invention will be described. However, these may be appropriately modified and combined. In the following description and the accompanying drawings, substantially the same or equivalent parts will be described with the same reference numerals. In the following description, the image will be described as including a moving image and a still image. Further, in the following, transmission or reception of information is an expression including transmission or reception of an information signal.

An information collecting system 100 that is Embodiment 1 of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is an overall configuration diagram of the information collecting system 100 according to the first embodiment. As shown in FIG. 1, the information collecting system 100 is configured by connecting a terminal device 10 as an information collecting device mounted on a vehicle M as a moving body and a server 50 to each other via a network NW. Although the automobile M is shown as an example of the moving body in FIG. 1, the moving body may be a moving body other than the automobile, such as a bicycle, a motorcycle, an airplane, a ship, or a moving person.

FIG. 2 is a diagram showing an example of attachment of the terminal device 10 to the automobile M. In FIG. 2, as an attachment example, the terminal device 10 attached to the front seat portion of the automobile M is shown.

The vehicle exterior photographing camera 11 as an external photographing unit for photographing the state of the vehicle M outside the vehicle (surroundings), that is, the outside of the vehicle M (the outside world) is arranged on the dashboard DB, for example. The vehicle exterior photographing camera 11 may be, for example, a wide-angle camera capable of photographing a wide area in front of the automobile M through the windshield.

The vehicle exterior photographing camera 11 may be provided anywhere as long as it can photograph the situation outside the vehicle. For example, the vehicle exterior photographing camera 11 may be provided on the back of the rearview mirror RM, that is, on the surface facing the windshield FG. Further, the vehicle exterior photographing camera 11 may be provided outside the automobile M, for example, on a front bumper or the like. Further, a plurality of vehicle exterior photographing cameras 11 may be provided so as to be able to photograph multiple directions outside the vehicle.

The in-vehicle image capturing camera 13 as the passenger image capturing unit is a camera that captures a situation such as an operation of the passenger as the passenger state information that is the passenger state information. The in-vehicle camera 13 is provided, for example, at the upper end of the windshield FG or at the ceiling near the upper end. The in-vehicle camera 13 may be provided anywhere as long as it is possible to take an image of the situation inside the vehicle and the passenger, and may be provided on the dashboard DB or the room mirror RM, for example.

The microphone 15 that receives the sound inside the vehicle as the passenger state information is arranged, for example, on the dashboard DB. The microphone 15 may be provided anywhere such as the interior mirror RM or the steering wheel as long as it can receive the sound inside the vehicle.

The in-vehicle camera 13 and the microphone 15 serve as an occupant status information collecting unit that collects occupant status information.

The GPS receiver 17 that receives a signal (GPS signal) from a GPS (Global Positioning System) satellite is arranged, for example, on the dashboard DB. The GPS receiver 17 may be provided anywhere as long as it can receive GPS signals.

The display 19 as a notifying means for displaying or notifying information received from the outside or the like is arranged, for example, on the dashboard DB. The display 19 should just be arrange|positioned in the place which can be visually recognized by the passenger. For example, the display 19 may be attached to the center console CC in the dashboard DB.

The main body 21 is connected to the vehicle exterior photographing camera 11, the vehicle interior photographing camera 13, the microphone 15, the GPS receiver 17, and the display 19, and includes a control unit (see FIG. 3) for controlling these. The main body 21 is mounted on, for example, the center console CC portion in the dashboard DB.

FIG. 3 shows a functional block diagram in the control unit 23 of the terminal device 10 and the server 50. First, the control unit 23 will be described.

The occupant monitoring unit 25 as a state change detection unit can receive information on the in-vehicle camera 13 and the microphone 15. The occupant monitoring unit 25 monitors the state of the occupant in the vehicle M, that is, the inside of the passenger compartment, based on the image or voice input transmitted from the in-vehicle photographing camera 13 and the microphone 15, and detects the state of the occupant. It is determined whether a change is detected, for example, whether an abnormality or an abnormality has occurred. When the passenger monitoring unit 25 determines that the change in the passenger's state has been detected, the passenger monitoring unit 25 transmits the state change detection information to the image extracting unit 27, the own position identifying unit 29, and the information transmitting unit 31.

The image extracting unit 27 is connected to the vehicle exterior photographing camera 11, and receives the image captured by the vehicle exterior photographing camera 11. The image extraction unit 27 extracts a part of the image received from the vehicle exterior photographing camera 11 based on the state change detection information from the occupant monitoring unit 25 to generate an extracted image. When the image extracting unit 27 generates the extracted image, the image extracting unit 27 captures the extracted image, that is, an event image obtained by photographing an event occurring outside the moving body when the state change is detected. Is transmitted to the information transmitting unit 31.

Note that the vehicle exterior photographing camera 11 does not always take a picture, the passenger monitoring unit 25 transmits the state change detection information to the vehicle exterior photographing camera 11, and the vehicle exterior photographing camera 11 takes a picture in response to the reception of the information. Good. For example, shooting may be performed when the state change detection information is received, or shooting may be performed for a certain period of time after the state change detection information is received. In this case, a part of the photographed image may be further extracted by the image extraction unit, or may be transmitted as an event image to the information transmission unit 31 without passing through the image extraction unit.

The own position specifying unit 29 is connected to the GPS receiver 17, and can output position specifying information indicating the current position of the automobile M in which the terminal device 10 is mounted. Upon receiving the state change detection information from the occupant monitoring unit 25, the own position specifying unit 29 specifies the position of the automobile M at the time when the state change detection information is received by the GPS signal from the GPS receiver 17, and detects the position of the position. The position specifying information, which is information, is transmitted to the information transmitting unit 31. Further, the own position specifying unit 29 can transmit the position specifying information to other parts such as the information receiving unit 33 as necessary.

The own position specifying unit 29 may specify the position of the vehicle M based on the posture information of the vehicle M from a gyro device (not shown) or the vehicle speed information obtained from the vehicle speed pulse of the vehicle M. In addition, the self-location specifying unit 29 may retain the map information or may be acquired from the outside. That is, even if at least one of the GPS information from the GPS receiver 17, the attitude information of the automobile M from the gyro device, and the vehicle speed information of the automobile M is combined with the map information, the position of the automobile M can be specified. Good.

The information transmission unit 31 as the external situation information generation unit or the external situation information transmission unit is connected to a transmitter (not shown) capable of transmitting an information signal to the server 50 via the network NW (see FIG. 1). ..

When the information transmission unit 31 receives the state change detection information from the passenger monitoring unit 25, the information transmission unit 31 generates information content as external situation information. The information transmitting unit 31 receives the event image from the image extracting unit 27 or the vehicle exterior camera 11, and also receives the position specifying information from the own position specifying unit 29. Then, based on these received information, the information content as the external situation information including the information of the reception time of the state change detection information (for example, the photographing time of the event image), the event image and the position identification information is generated. That is, the information transmission unit 31 generates information content according to the state change detection information.

After generating the information content, the information transmitting unit 31 transmits the information content to the server 50. Even if the state change detection information is not directly received from the passenger monitoring unit 25, the state change detection information may be sent from the passenger monitoring unit 25 to the information transmitting unit 31 upon reception of the event image or the position specifying information. It is okay to assume that you have been invited.

The information receiving unit 33 is connected to a receiver (not shown) capable of receiving information from the outside such as the server 50 via the network NW. Upon receiving the information from the server 50, the information receiving unit 33 can display the information on the display 19 based on the information and notify the passengers of the information based on the information.

Next, the server 50 will be described. The information receiving unit 51 as the external situation information receiving unit can receive the information content from the information transmitting unit 31 via the network NW. When receiving the information content, the information receiving unit 51 transmits it to the content selection unit 53. Although not shown, the information receiving unit 51 receives the information content from the plurality of terminal devices 10 mounted on the other vehicles M, respectively.

The content selection unit 53 as a selection unit or a determination unit is a functional unit that receives the information content and selects the information content to be transmitted to the automobile M from the information content. The content selection unit 53 includes a content usefulness determination unit 55 that determines whether or not the information content is useful or significant from the viewpoint of contributing to selection of the travel route of the automobile M from the information contents.

The content usefulness determination unit 55, for example, when a plurality of information contents are received and accumulated, among the accumulated information contents, information contents having position specifying information indicating positions that are the same or close to each other (hereinafter, It is determined whether there are a large number (or a predetermined number or more) of neighboring contents). When there are a predetermined number or more of nearby content, the content usefulness determination unit 55 determines that the nearby content group is useful or significant, selects these, and transmits them to the representative content selection unit 57.

Note that the near-field content is information content that has, among information content, position specifying information that indicates positions that are the same as or near each other and that has event images that have been shot during the same time period (for example, the shooting time is within 30 minutes before and after). May be

The content selection unit 53 includes a representative content selection unit 57 that selects, as the representative content, the information content that holds the most useful or significant or high-quality event image from the neighboring content group. The representative content selection unit 57 may analyze, for example, the event images held by the neighboring content group and select the information content holding the clearest event image as the representative content. Moreover, you may select the information content which hold|maintains the event image image|photographed recently as a representative content from the neighborhood content group.

When the representative content selecting unit 57 selects the representative content, it transmits it to the information transmitting unit 59.

The information transmission unit 59 as an external situation information distribution unit can transmit an information signal to the terminal device 10 via the network NW. Upon receiving the representative content from the representative content selection unit 57, the information transmission unit 59 transmits it to the terminal device 10.

As described above, when the information receiving unit 33 of the terminal device 10 receives the information from the server 50, the information receiving unit 33 can cause the display 19 to display the information based on the information and notify the information. This display is performed, for example, by displaying the position of the automobile M and the position specifying information included in the information content, that is, the position where the event has occurred, and the event image included in the information content on the map.

Here, after receiving the representative content, the information receiving unit 33 determines whether to display the information based on the representative content on the display 19 based on the position specifying information of the representative content and the shooting time of the event image. It may be that.

This determination may be performed, for example, based on whether or not the current position of the automobile M equipped with the terminal device 10 and the position indicated by the position specifying information included in the representative content are within a predetermined distance.

When the navigation system is installed in the automobile M, it is determined whether the route set in the navigation system passes through the position indicated by the position specifying information of the representative content or in the vicinity thereof. May be done.

[Operation routine of each part]
The operation routines of the above-mentioned units will be described below. In addition, among the operation routines described below, the operation routine of each unit in the control unit 23 of the terminal device 10 is started, for example, at the time of starting a prime mover such as an engine and a motor of the automobile M, and is repeatedly executed until the prime mover is stopped. It Further, an individual switch of the terminal device 10 may be provided, and the operation routine may be started when the switch is turned on and repeatedly executed until the switch is turned off.

FIG. 4 shows an example of the operation routine of the passenger monitoring unit 25. First, the occupant monitoring unit 25 determines whether the occupant has undergone a state change, for example, an abnormality (step S1). In this determination, for example, whether the state change has occurred based on whether the gaze state of the operator or the passenger has changed, specifically, whether the gaze state has changed. May be determined. The change in the gaze state includes, for example, a change from a state of no gaze to a state of gaze, that is, a change of the degree of gaze or a change of the direction of gaze.

For example, when it is detected by the line-of-sight analysis or the like using the image transmitted from the in-vehicle photographing camera 13 that the driver gazes in one direction for a certain period of time or more, it is determined that a state change has occurred (is detected). May be

Further, it may be determined that the state change has occurred when it is detected that a large number of passengers including the pilot gaze in one direction for a certain period of time or longer. Even if the line-of-sight analysis is not performed, it may be determined that the driver or the passenger is gazing at the one direction when the direction of the passenger's face including the driver faces the one direction for a certain time or more. ..

Also, when it is detected that the movement of the operator's gaze direction (position) or gaze direction exceeds the threshold value, or when the gaze direction (position) or gaze direction moves at a speed exceeding the threshold value. It may be determined that the state change has occurred.

Also, the voice transmitted from the microphone 15 is voice-recognized, and when a specific keyword such as “accident”, “construction”, “difficult”, “what is it?” is recognized, it is determined that an abnormality has occurred It may be that. Note that the state change detection information may include gaze direction information that is information on the direction in which the driver or the passenger gazes when the state change occurs. Further, the state change detection information may include the result of voice recognition.

When it is determined in step S1 that a state change has occurred (step S1: YES), the occupant monitoring unit 25 transmits the state change detection information to the image extracting unit 27, the own position identifying unit 29, and the information transmitting unit 31. (Step S2), the operation routine ends. When it is determined in step S1 that the state has not changed (step S1: NO), the operation routine ends thereafter.

FIG. 5 shows an example of the operation routine of the image extraction unit 27. First, the image extraction unit 27 determines whether the state change detection information has arrived from the passenger monitoring unit 25 (step S1). When it is determined in step S1 that the state change detection information has not arrived (step S1: NO), the operation routine ends thereafter.

When it is determined in step S1 that the state change detection information has arrived, the image extraction unit 27 extracts the image of the vehicle exterior camera 11 to generate an extracted image (step S2). The generation of this extracted image may be performed by extracting a still image at the time when the state change detection information is received, from the image of the vehicle exterior photographing camera 11.

Alternatively, one or a plurality of still images before and after the time when the state change detection information is received or a moving image in a predetermined time range before and after the time may be extracted from the image of the vehicle exterior camera 11. Further, as described above, when the state change detection information includes the gaze direction information of the passenger, the still image extracted based on the time when the state change detection information is received based on the gaze direction information or The partial area may be further cut out (trimmed) from the moving image. At this time, the image extraction unit functions as an image cutout unit.

In this case, an area portion that matches the gaze direction of the passenger may be cut out and extracted from the image or the moving image extracted based on the time when the state change detection information is received, based on the gaze direction information.

After the end of step S2, the image extraction unit 27 uses the extracted image as an image when the state change is detected, that is, an event image obtained by capturing an event occurring outside the moving body when the state change is detected. , To the information transmitting unit 31 (step S3), and then the operation routine ends.

Note that, as described above, the vehicle exterior camera does not always take a picture, and the occupant monitoring unit 25 transmits the state change detection information to the vehicle exterior camera 11, and the vehicle exterior camera 11 takes a picture in response to the reception of the information. You may go. For example, shooting may be performed when the state change detection information is received, or shooting may be performed for a certain period of time after the state change detection information is received.

In this case, the captured image may be temporally extracted by the image extraction unit 27 and transmitted to the information transmission unit 31 as an event image. In addition, the captured image may be cut out by the image extraction unit 27 that functions as an image cutout unit, and a partial region of the image may be cut out and transmitted to the information transmission unit 31 as an event image. The captured image may be directly transmitted as an event image to the information transmitting unit 31 without passing through the image extracting unit 27.

FIG. 6 shows an example of the operation routine of the own position specifying unit 29. First, the own position identifying unit 29 determines whether the state change detection information has arrived from the passenger monitoring unit 25 (step S1). When it is determined in step S1 that the state change detection information has not arrived (step S1: NO), the operation routine ends thereafter.

When it is determined in step S1 that the state change detection information has arrived, the self-position specifying unit 29 generates position specifying information that specifies the position of the automobile M on which the terminal device 10 is mounted (step S2). The position specifying information may be information on the position specified by the GPS signal from the GPS receiver 17. The own position specifying unit 29 may generate position specifying information for specifying the position of the vehicle M based on the attitude information of the vehicle M or a vehicle speed pulse of the vehicle M from a gyro device (not shown).

In addition, the self-location specifying unit 29 may retain the map information or may be acquired from the outside. That is, the self-position specifying unit 29 combines the map information with at least one of the GPS signal from the GPS receiver 17, the attitude information of the vehicle M from the gyro device, and the vehicle speed information of the vehicle M to determine the position of the vehicle M. You may generate the position specific information which specifies.

After the end of step S2, the own position specifying unit 29 transmits the generated position specifying information to the information transmitting unit 31 (step S3), and then the operation routine ends.

The event image and the position specifying information transmitted to the information transmitting unit 31 are transmitted to the information receiving unit 51 of the server 50 by the information transmitting unit 31 as information contents including these. That is, the information transmitting unit 31 generates the information content including the event image and the position specifying information, and transmits the information content to the information receiving unit 51 of the server 50. Upon receiving the information content, the information receiving unit 51 transmits the information content to the content selection unit 53.

FIG. 7 shows an example of an operation routine of the content selection unit 53 including the content usefulness determination unit 55 and the representative content selection unit 57. In this example, the content selection unit 53 determines whether or not a certain number or more of the information contents including the event images captured at the positions within the certain range have arrived in order to determine whether the information contents are significant or useful. judge. As described above, the information receiving unit 51 receives the information content from the plurality of terminal devices 10 mounted on the plurality of automobiles M, respectively.

First, the content selection unit 53 determines whether or not a predetermined number or more of information contents (hereinafter, also referred to as neighboring contents) having position specifying information indicating positions that are the same as or near each other have arrived from the information receiving unit 51 (step). S1). When it is determined in step S1 that the predetermined number or more of neighboring contents have not arrived (step S1: NO), the operation routine is ended thereafter.

When it is determined in step S1 that the number of neighboring contents has reached a predetermined number or more (step S1: YES), the content sorting unit 53 sorts one information content from the neighboring contents as a representative content (step S2). ..

This representative content may be selected, for example, by analyzing the event images held by each of the neighboring contents, using the clearest event image as the representative image, and selecting the information content holding the representative image as the representative content. Further, among the event images held by each of the neighboring contents, the event image captured most recently may be used as the representative image, and the information content holding the representative image may be selected as the representative content.

The information content may include the vehicle speed pulse of the automobile M at the time of capturing the event image or the vehicle speed information based on the vehicle speed pulse. In that case, the vehicle speed pulse or the vehicle speed information is used to select the clearest event image. It may be used. For example, the vehicle speed pulse at the time of capturing the event image may be analyzed, the event image captured at the lowest speed may be used as the representative image, and the information content holding the representative image may be used as the representative content.

Further, the information content includes information on the direction (position) or the line-of-sight direction in which the occupant of the vehicle M is gazing at the time of transmitting the state change detection information from the occupant monitoring unit 25 (that is, at the time of capturing the event image). May be included. In this case, for example, it may be determined from the event image which lane the vehicle M was traveling (stopped in) on the road on which the event image was captured, and the representative image may be determined. ..

For example, when the passenger is gazing in the left direction of the automobile M when transmitting the state change detection information, the event image captured in the leftmost lane may be used as the representative image. Further, when the passenger is gazing at the right side of the automobile M, the event image captured in the rightmost lane may be used as the representative image.

When step S2 ends, the content selection unit 53 transmits the representative content that is the information content including the representative image to the information transmission unit 59 (step S3), and then the operation routine ends. Upon receiving the information content from the content selection section 53, the information transmission section 59 transmits it to the information reception section 33 of the terminal device 10.

FIG. 8 shows an example of the operation routine of the information receiving unit 33. First, the information receiving unit 33 determines whether the information content has arrived from the server (step S1). When it is determined that the information content has not arrived (step S1: NO), the operation routine ends thereafter.

When it is determined in step S1 that the information content has arrived from the server (step S1: YES), the information receiving unit 33 determines whether the arrived information content is useful to the passenger of the automobile M in which the terminal device 10 is mounted. It is determined (step S2).

In this determination, for example, when the position indicated by the position specifying information included in the arrived information content is near the current position of the automobile M (within a predetermined distance), the information content is determined to be useful. You may.

Further, for example, when a route is set in the navigation system mounted on the automobile M, it may be determined whether or not the information content arrived based on the route is useful. That is, for example, the position indicated by the position identification information held by the information content that has arrived is on the set route or in the vicinity of the route, and the information content is useful. You may decide to judge.

When it is determined in step S2 that the received information content is useful (step S2: YES), the information receiving unit displays information based on the information content on the display 19 (step S3). This display may simply display the event image and the position specifying information held by the information content, for example. In addition to this, the position where the event image is captured may be displayed on the map based on the position specifying information.

If it is determined in step S2 that the incoming information content is not useful (step S2: NO), then the operation routine ends. When step S3 ends, the operation routine ends thereafter.

According to the information system 100, the terminal device 10 and the server 50 described above, an unspecified position on the moving route of the moving object, for example, a narrow road or a road with a small traffic volume, includes a route where the information collecting device is not installed. It is possible to collect information, including images, of what happened at the location. Furthermore, it is possible to deliver the collected information to other mobile bodies.

Further, according to the information system 100, the terminal device 10, and the server 50, information including an image captured at the time of the state change is generated based on a state change such as an abnormality of the passenger of the moving body, and the image is included. By analyzing information and selecting useful or significant information and delivering it, only necessary information can be delivered efficiently.

A system 200 according to the second embodiment of the present invention will be described below with reference to the drawings. The system 200 of the second embodiment has the same configuration as the system 100 of the first embodiment except that the operation routine of the content selection unit 53 is different. Therefore, explanations other than the operation routine of the content selection unit 53 are omitted.

In the second embodiment, the content selection unit 53 of the system 200 analyzes the event image in the information content transmitted from the terminal device 10 to analyze what event the event image was captured, and An example will be described in which the significance or usefulness of the information content is determined. That is, a case will be described as an example where the content selection unit 53 analyzes what kind of event is occurring at the time of capturing the event image and determines the significance or usefulness.

FIG. 9 shows an example of the operation routine of the content selection unit 53 of the system 200 of the second embodiment. First, the content selection unit 53 determines whether the information content has arrived from the terminal device 10 (step S1). When it is determined in step S1 that the information content has not arrived (step S1: NO), the operation routine ends thereafter. In addition, as described above, the information content comes from the plurality of terminal devices 10 mounted in the plurality of automobiles M, respectively.

When it is determined in step S1 that the information content has arrived (step S1: YES), the content selection unit 53 determines whether the arrived information content is useful or significant (step S2).

This determination is performed, for example, by analyzing what the event image included in each of the information contents was captured, that is, what kind of event is occurring. For example, if the event image is analyzed to be a useful image such as “accident”, “construction site”, “traffic jam”, “crowd”, “event (event)”, the event is analyzed. The information content containing the images may be determined to be useful or significant.

Specifically, for example, the event image is image-analyzed, and when it is analyzed that a large number of automobiles have not moved for a certain period of time, it is determined that the traffic jam. If the letters "construction" are recognized or the yellow and black diagonal lines seen on the construction site are recognized, it is determined as "construction site". In addition, for example, when a large number of people are recognized, it is determined to be “crowded” or “event”. If it is determined in step S2 that the information content is not useful or significant (step S2: NO), the routine ends thereafter.

When it is determined in step S2 that the arrived information content is useful or significant (step S2: YES), the process proceeds to step S3. In step S3, the content selection unit 53 photographs at the same time zone (for example, the photographing time is within 30 minutes before and after) and at the same position (or near position) as the event image held by the information content determined to be useful or significant. It is determined whether or not another information content including the selected event image has arrived (step S3).

When it is determined in step S3 that another information content including the event image captured at the same time zone and at the same position has arrived (step S3: YES), the content selection unit 53 determines that the information content One information content is selected as a representative content from the inside (step S4).

As described in the first embodiment, the selection of the representative content is performed by, for example, analyzing the event images held by each of the information contents, setting the clearest event image as the representative image, and selecting the information content holding the representative image. You may select as a representative content. Further, the event image captured most recently among the event images held by the information content may be used as the representative image, and the information content holding the representative image may be selected as the representative content.

It should be noted that the information content may include information on the vehicle speed pulse or vehicle speed of the automobile M at the time of capturing the event image. In that case, the vehicle speed pulse or vehicle speed information is used to select the clearest event image. May be. For example, the vehicle speed pulse at the time of capturing the event image may be analyzed, the event image captured at the lowest speed may be used as the representative image, and the information content holding the representative image may be used as the representative content.

Further, the information content includes information on the direction (position) or the line-of-sight direction in which the occupant of the vehicle M is gazing at the time of transmitting the state change detection information from the occupant monitoring unit 25 (that is, when capturing the event image). May be included. In this case, for example, it may be possible to determine which lane the automobile M is traveling (stop in which lane) from the event image and determine the representative image.

For example, when the passenger is gazing in the left direction of the automobile M when transmitting the state change detection information, the event image captured in the leftmost lane may be used as the representative image. In addition, when the passenger is gazing at the left side of the automobile M, the event image captured in the rightmost lane may be used as the representative image.

After step S4 and in step S3, when it is determined that no other information content including the event image captured at the same time zone and the same position has arrived (step S3: NO), the content selection unit 53 transmits the information content including the representative image as it is to the terminal device 10 (step S5).

When it is determined in step S3 that no other information content including the event images captured at the same time zone and the same position has arrived, the information content determined to be useful or significant in step S2 is set to the terminal device 10. Send to. After step S5 ends, the operation routine ends.

In this case, the information content transmitted from the server 50 to the terminal device 10 may be the one to which the analysis result of the event image performed in step S2 is added. That is, the information content to be transmitted includes information on the event shown in the event image (for example, “accident”, “construction site”, “traffic jam”, “crowd”, or “event”). It may be done.

When the information related to such an event is included in the information content and transmitted to the terminal device 10, the information receiving unit determines whether to display the information based on the information content on the display 19 based on the information about the event. You may judge. For example, when the user who uses the terminal device 10 requests only the information of “accident”, only the information based on the information content regarding “accident” may be displayed on the display 19.

When the automobile M is equipped with a navigation system, the navigation system may be provided with information regarding the event to prompt the navigation system to change the route. For example, information that an accident has occurred on the set route of the navigation system may be provided based on the information content, and the set route may be changed so as to avoid the accident.

Further, for example, based on the information content, information that an “event (event)” such as fireworks is being held near the set route of the navigation system is provided, and the set route is set to the “event (event)”. It may be possible to encourage a change so as to pass through a place where "event" is held. In addition, in consideration of congestion and congestion due to “events (events)”, it is possible to prompt a change of the set route so as to avoid the place where the “events (events)” are being performed.

According to the information system 200, the terminal device 10 and the server 50 described above, an unspecified position on the moving route of the moving body, for example, a narrow road or a road with a small traffic volume, includes a route where the information collecting device is not installed. It is possible to collect information, including images, of what happened at the location. Furthermore, it is possible to deliver the collected information to other mobile bodies.

Further, according to the information system 200, the terminal device 10 and the server 50, information including an image captured at the time of the state change is generated based on a state change such as an abnormality of the passenger of the moving body, and the image is included. By analyzing information and selecting useful or significant information and delivering it, only necessary information can be delivered efficiently.

Further, according to the information system 200, the terminal device 10 and the server 50, it is possible to analyze an image of an event captured and transmitted in a mobile body, and deliver information including information about what kind of event is occurring. is there. As a result, detailed information regarding the event that has occurred can be distributed to other moving bodies such as automobiles.

Below, the system 300 of Example 3 is demonstrated with reference to FIG. As shown in FIG. 10, in the system 300, the passenger monitoring unit 25 and the image extracting unit 27, which are provided on the terminal device 10 side in the first and second embodiments, serve as the passenger monitoring unit 25S and the image extracting unit 27S. , Provided on the server 50 side. That is, the so-called cloud configuration is configured so that the server side bears the processing on the terminal device 10 side.

In the system 300, the image and sound of the vehicle exterior camera 11, the vehicle interior camera 13, and the microphone 15, and the position specifying information from the own position specifying unit 29 are constantly transmitted by the information transmitting unit 31 to the server 50 via the network NW. ing. The transmitted image and sound are received by the information receiving unit 51 of the server 50. The above-mentioned image, sound, and position identification information come from a plurality of terminal devices 10 mounted on a plurality of automobiles M, respectively.

As for the received image and sound, the image and sound of the in-vehicle shooting camera 13 and the microphone 15 are sent to the passenger monitoring unit 25S, and the image of the out-of-vehicle shooting camera 11 and the position specifying information from the own position specifying unit 29 are sent to the image extracting unit 27S. Each is sent. In this transmission, the image, the sound, and the position specifying information may be transmitted in a form that can be identified for each vehicle M that is the transmission source.

In the system 300, the occupant monitoring unit 25S executes the same operation routine as the occupant monitoring unit 25 of the first embodiment shown in FIG. 4, for example, to monitor the condition of the occupant. The image extracting unit 27S executes the same operation routine as that of the image extracting unit 27 of the first embodiment shown in FIG. 5, for example. Note that the image extracting unit 27S transmits the shooting time of the event image and the position specifying information together with the extracted event image to the content selection unit 53 in step S3 of FIG.

The content selection unit 53 executes, for example, the same operation routine as that of the first or second embodiment (for example, the operation routine shown in FIG. 7 or 9) and selects the information content to be transmitted to the terminal device 10. Then, the selected information content is transmitted from the information transmitting unit 59.

The transmitted information content is received by the information receiving unit 33. The information receiving unit 33 executes, for example, the same operation routine as that of the first or second embodiment (for example, the operation routine shown in FIG. 8). The information receiving unit 33 displays the information on the display 19 as needed in the same manner as that described in the first or second embodiment.

According to the system 300 of the third embodiment, in addition to the advantages of the first or second embodiment, it is possible to reduce the processing load of the terminal device 10 and reduce the power consumption of the terminal device 10. Further, the terminal device 10 can be downsized.

In the above-described embodiments, the wide-angle camera is taken as an example of the vehicle exterior photographing camera 11, but various other cameras can be used. For example, a camera provided on a mounting table with an adjustable angle and capable of changing the shooting direction may be used. In this case, based on the gaze direction information in the state change detection information transmitted by the occupant monitoring unit 25, the shooting direction of the vehicle exterior camera 11 is changed to the gaze direction and shooting is performed, and the shot image is taken. Part or all of the above may be used as the event image.

Further, in the above-described embodiment, the passenger monitoring unit 25 determines that the passenger inside the vehicle has changed state based on the image of the vehicle interior photographing camera 13 and the voice of the microphone 15 as the passenger state information collecting unit. However, the determination may be performed using information other than this.

For example, a biometric sensor that detects biometric information (vital sign) of the occupant is provided as the occupant status information collecting unit, and the occupant in the vehicle is based on the information from the biosensor (for example, change in heart rate or blood pressure). The state change detection information may be transmitted after determining that the state change has occurred. This biometric sensor may be attached to, for example, an occupant, or may be provided on a steering wheel, a seat and a seat belt.

Further, as the occupant status information collecting unit, a unit for collecting information regarding a driving operation on the automobile M may be provided. In this case, the occupant monitoring unit 25 may detect operations such as sudden acceleration, sudden braking, and sudden steering, and transmit the state change detection information based on these operations.

Further, in order to reduce detection of unnecessary state change, for example, when the shift lever is moved to the back position, the state change may be detected in consideration of the situation. For example, when the shift lever is moved to the back position, the backward gaze of the automobile M may not be detected as a state change.

Further, in the above-described embodiment, the terminal device 10 has the display 19 as the notification means, but it may include a speaker as the notification means. In this case, the notification based on the information content may be given by the display on the display 19 and the voice from the speaker.

Further, the notification based on the information content may be made only by voice. In the case of notification only by voice, the position specifying information included in the information content, that is, the occurrence position of the event, the occurrence time of the event, and the content of the event may be notified by voice.

Further, in the above-described embodiment, the content selection unit 53 selects one information content as the representative content from the neighboring contents, but a plurality of information contents may be selected as the representative content. In this case, for example, a plurality of information contents may be selected as the representative contents in the order in which the included event images are clear.

Further, in the above-described embodiment, the terminal device 10 has the information receiving unit 33 and the display 19, but the terminal device 10 may not include the information receiving unit 33 and the display 19. That is, the terminal device 10 may be an information collecting device that only transmits information to the server 50.

Further, in the above-described embodiment, the case where the representative content is transmitted to all the mobile bodies M has been described as an example, but the representative content may be transmitted to some of the automobiles M. For example, the server 50 holds the position information of each vehicle M, and only the vehicle M located in the vicinity of the position indicated by the position specifying information included in the representative content, that is, within a certain distance from the position indicated by the position specifying information. It may be sent to.

Further, in the above-mentioned embodiment, the information content includes the position specifying information indicating the position information of the automobile M, but the position specifying information may not be included. In this case, the server 50 may specify the position of the automobile M from information such as which base station of the mobile communication network the wireless communication from the automobile M is through. Further, the position of the event image may be specified from the event image included in the information content.

Further, when the vehicle travels to the position indicated by the position specifying information included in the representative content, the representative content is limited to only the automobile M existing in the position where the arrival time to the position indicated by the position specifying information is within a certain time. May be transmitted.

The information content from the server 50 may be receivable by a device other than the terminal device 10, such as a smartphone, a tablet, a PDA, and a personal computer.

Further, in the above-described embodiment, the case where the terminal device 10 is mounted on the automobile M as an example of the moving body has been described as an example, but the moving body is not limited to the automobile. For example, the moving body on which the terminal device 10 is mounted may be a ship, an airplane, a motorcycle, a bicycle, or the like.

A person moving on foot may hold the terminal device 10 and move, and the terminal device 10 may transmit the information content to the server 50. At this time, the in-vehicle camera as the passenger photographing means of the terminal device 10 becomes a camera for photographing the moving person.

For example, the terminal device 10 may be realized by a smartphone or a tablet. In that case, the inside camera of the smartphone or tablet may correspond to the in-vehicle photographing camera described in the first to third embodiments, and the outer camera may correspond to the outside-vehicle photographing camera described in the first to third embodiments.

Moreover, in the said Example, although the structure of the terminal device 10 and the server 50 was illustrated and demonstrated to FIG. 3 and FIG. 10, the structure of the terminal device 10 and the server 50 is not restricted to this example. For example, the passenger monitoring units 25 and 25S, the image extraction units 27 and 27S, and the content selection unit 53 may be arranged in either the terminal device 10 or the server 50.

The various configurations, routines, and the like in the above-described embodiments are merely examples, and can be appropriately selected according to the application and the like.

100, 200, 300 Information collecting system 10 Terminal device 11 Vehicle exterior camera 13 Vehicle interior camera 15 Microphone 17 GPS receiver 19 Display 21 Main body 23 Control unit 25, 25S Passenger monitoring unit 27, 27S Image extraction unit 29 Self-location specifying unit 31 information transmission unit 33 information reception unit 50 server 51 information reception unit 53 content selection unit 55 content usefulness determination unit 57 representative content selection unit 59 information transmission unit

Claims (1)

An occupant status information collecting unit for collecting the occupant status of the moving body as occupant status information;
A state change detection unit that detects a state change of the passenger based on the passenger state information and generates state change detection information;
An external photographing section for photographing the outside of the moving body to generate a photographed image;
An external situation information transmission unit that generates external situation information including the captured image according to the state change detection information and transmits the external situation information via a communication path;
An information collection device comprising:

Images