JP5395716B2 - Outside sound providing device, outside sound providing method and program - Google Patents

Description

  The present invention relates to an apparatus for providing sound outside a vehicle to a passenger in the vehicle.

  When a car is closed, sound outside the car is blocked. When the outside of the vehicle is noisy, you can escape from the fight outside the vehicle by closing the window. On the other hand, when an event or activity is performed outside the vehicle, the occupant is isolated from the atmosphere of the event or the like, and cannot enjoy the enjoyment of driving. Of course, it is possible to capture sound outside the vehicle by opening the window, but it is not preferable to open the window when using an air conditioner in the vehicle.

  As an apparatus that collects sound outside the vehicle with a microphone and provides it to the occupant, there is an invention of an information providing apparatus outside the vehicle described in Patent Document 1. In this invention, when a mobile sound source such as an overtaking vehicle is detected using a plurality of microphones, a sound collection directivity control technology, a mobile sound source detection technology, and a vehicle motion information acquisition technology, The sound is collected and provided to passengers.

JP 2009-258802 A

  Conventionally, there has been no device that captures the sound outside the vehicle in order to achieve a sense of unity between the outside and the inside of the vehicle and improve the quality of the drive. Patent Document 1 is an invention that provides a driver with sound outside the vehicle. The purpose of the present invention is to improve driving safety, and the approach of an overtaking vehicle or an emergency vehicle that exists outside the field of view. Communicate to the driver. Therefore, this device does not improve the drive quality.

  In view of the above background, an object of the present invention is to provide a vehicle exterior sound providing apparatus that can enhance the sense of unity between the vehicle exterior and the vehicle interior.

  A vehicle exterior sound providing apparatus according to the present invention includes a camera that captures the outside of a vehicle, a person detection unit that detects a person outside the vehicle, a directional microphone that collects sounds outside the vehicle, and an image captured by the camera other than a road. A sound collection direction determination unit that detects a target region to be collected from regions other than the road based on the detection result by the person detection unit, and determines the direction of the target region as the sound collection direction And a directional microphone control unit that directs the directional microphone in the sound collection direction, and a speaker that outputs the sound collected by the directional microphone into the vehicle. Here, the camera may be an infrared camera, and the person detection unit may detect a heat source having a predetermined size or more as a person from an image taken by the infrared camera.

  In this way, by determining the sound collection direction using the area other than the road as the target area, the sound of activities and events taking place outside the vehicle is provided inside the vehicle, not the sound related to driving support, and the quality of the drive Can be increased. Further, by obtaining the target area based on the detection result by the person detection unit, the direction in which the person is detected or the direction in which no person is detected can be obtained as the sound collection direction. As a result, for example, when you are in a city, you can collect the sounds and voices that people are listening to by collecting sound from the direction in which the person was detected. If you are in nature, the person is detected. Sounds that are appropriate for the scene can be collected, such as collecting the sound of waves and the voice of insects from the direction that was not there.

  The vehicle exterior sound providing apparatus of the present invention includes an input unit that receives selection of whether the direction in which the person is present is the sound collection direction or the direction in which no person is the sound collection direction, and the sound collection direction determination unit is Depending on the selection, the direction in which the person is detected or the direction in which the person is not detected may be determined as the sound collection direction. Thereby, the vehicle exterior sound which a passenger | crew desires can be provided.

  The vehicle exterior sound providing apparatus of the present invention includes a vehicle speed data acquisition unit that acquires vehicle speed data from a speed sensor that detects the vehicle speed, and does not provide vehicle exterior sound when the vehicle speed is equal to or greater than a predetermined threshold. Also good. With this configuration, when the vehicle speed is equal to or higher than a predetermined threshold, an environment where the user can concentrate on driving can be created.

  In the vehicle exterior sound providing apparatus of the present invention, the sound collection direction determination unit may not include the person area in the target area when the detected person is moving or when the detected person is alone. It is good. When the detected person is moving or alone, there is a possibility that he / she is not talking. Therefore, the directional microphone can be pointed in an appropriate direction by not using the target area.

  The vehicle exterior sound providing apparatus according to the present invention includes a position data acquisition unit that acquires data of a current position of a vehicle, and a road map database that stores a road map including facility data, and the sound collection direction determination unit includes: Based on the current position of the vehicle acquired by the position data acquisition unit and the data stored in the road map database, it is determined whether or not the person area is within a predetermined range including a facility. If it is not within the predetermined range, the person area may not be included in the target area. With this structure, conversations that are likely to be related to events and activities around the facilities described above by collecting sound around facilities such as stores, stadiums, and sightseeing spots, but not collecting sound at other locations. Can be collected properly.

  In the vehicle exterior sound providing apparatus of the present invention, the sound collection direction determination unit may obtain a target area based on a person area within a predetermined distance from the vehicle. As a result, it is possible to appropriately collect the utterance contents targeting a person who is speaking to the host vehicle.

  In the vehicle exterior sound providing apparatus of the present invention, the directional microphone control unit may reduce the sensitivity of the directional microphone when the number of persons in the target area is equal to or less than a predetermined threshold. In the case of a small group, there is a possibility of having a private conversation, but with the configuration of the present invention, sound can be collected in a manner that does not infringe on privacy.

  A vehicle exterior sound providing apparatus according to the present invention includes a position data acquisition unit that acquires data of a current position of a vehicle, a road map database that stores a road map including data indicating a region classification, a region classification, and a directional microphone. A microphone sensitivity table that stores the sensitivity in association with each other, and the directional microphone control unit reads out and reads out from the road map database a region classification corresponding to the current position of the vehicle acquired by the position data acquisition unit. The sensitivity of the directional microphone corresponding to the classified area may be read from the microphone sensitivity table, and the directional microphone may be controlled so as to collect sound with the read sensitivity. For example, in a quiet area, there is a possibility of collecting not only the voice of people outdoors but also the voice of people indoors. By changing, sound can be collected in a manner that does not infringe on privacy.

  A vehicle exterior sound providing apparatus according to the present invention provides a position data acquisition unit that acquires data on the current position of a vehicle, a road map database that stores a road map that includes data indicating the type of road, a road type, and a vehicle exterior sound. A provision availability table storing data relating to whether to perform or not, and reading out the road type corresponding to the current position of the vehicle acquired by the position data acquisition unit from the road map database, Whether or not to provide outside sound may be determined based on the type and the data stored in the provision availability table. With this configuration, it is possible to appropriately determine whether or not to collect sound depending on the type of road. For example, on highways, it is possible not to interfere with safe driving by stopping the provision of external sound.

  In the vehicle exterior sound providing apparatus of the present invention, the sound collection direction determining unit may not include a person area in an area opposite to the host vehicle with the opposite lane in between as target areas. By excluding the area on the opposite side across the oncoming lane from the sound collection direction, it is possible to reduce the sound collection noise caused by the oncoming vehicle.

  The vehicle exterior sound providing apparatus according to the present invention is configured to output a vehicle exterior sound from the speaker based on a vehicle state data acquisition unit that detects a vehicle state and vehicle state data acquired by the vehicle state data acquisition unit. A provision timing determining unit that determines With this configuration, it is possible to provide vehicle exterior sound at a timing that ensures the safety of the vehicle.

  In the vehicle exterior sound providing apparatus of the present invention, the vehicle state data acquisition unit acquires vehicle speed data from a speed sensor that detects the speed of the vehicle, and the provision timing determination unit is configured to accelerate or decelerate the vehicle, or When the acceleration / deceleration is larger than a predetermined threshold, it may be determined not to output the outside sound. When the vehicle is accelerating or decelerating, it is necessary to be more careful than usual, so that it is possible to ensure the safety of the vehicle by not providing outside sound. Further, when the degree of acceleration / deceleration is greater than a predetermined threshold value, the configuration in which vehicle exterior noise is not provided does not hinder the provision of vehicle exterior noise due to slight acceleration / deceleration that is not intended by the driver. In addition, it is good also as providing later the vehicle outside sound which was not provided during acceleration / deceleration.

  In the vehicle exterior sound providing apparatus of the present invention, the vehicle state data acquisition unit acquires data indicating a shift position from a shift sensor that detects a shift position, and the providing timing determination unit has a shift in a “reverse” position. Sometimes, it may be determined not to output the sound outside the vehicle. Since caution is required when the vehicle is moving backward, it is possible to ensure vehicle safety by not providing outside sound. In addition, it is good also as providing the exterior sound which was not provided during the reverse drive later.

  When the directional microphone collects sound from a plurality of sound collecting directions, the vehicle exterior sound providing apparatus of the present invention may select the sound to be provided in the vehicle based on the volume of the collected sound. This configuration can provide a highly valuable sound.

  When the directional microphone collects sound from a plurality of sound collecting directions, the vehicle exterior sound providing apparatus of the present invention may select sound to be provided in the vehicle based on the frequency of the collected sound. Since it is possible to determine whether or not a person's voice is included in the sound collected based on the frequency, it is possible to provide a sound including or not including a person's voice according to the needs of the occupant.

  The vehicle exterior sound providing method of the present invention is a method of providing vehicle exterior sound into the vehicle, wherein the vehicle exterior sound providing device acquires an image outside the vehicle photographed by a camera, and the vehicle exterior sound providing device comprises: A step of acquiring data of a person outside the vehicle detected by a person detection unit; and the vehicle exterior sound providing device detects an area other than a road from an image photographed by the camera, and A target area to be collected based on a detection result by the person detection unit, determining a direction of the target area as a sound collection direction, and the vehicle exterior sound providing apparatus using a directional microphone A step of collecting sound from a sound direction, and a step of outputting the sound collected by the directional microphone into the vehicle by a speaker. With this configuration, as in the above-described vehicle exterior sound providing device, sounds such as activities and events performed outside the vehicle can be provided inside the vehicle, and the quality of the drive can be improved. Various configurations of the above-described vehicle exterior sound providing apparatus can be applied to the vehicle exterior sound providing method of the present invention.

  The program of the present invention is a program for providing sound outside the vehicle to the inside of the vehicle, wherein a camera that captures the outside of the vehicle, a directional microphone that collects sound outside the vehicle, and a speaker that outputs sound inside the vehicle are connected. A step of acquiring an image outside the vehicle photographed by the camera, a step of detecting a person outside the vehicle from the image outside the vehicle, and an area other than the road from the image photographed by the camera. Detecting a target area to be collected from areas other than roads based on the detection result of the person, determining a direction of the target area as a sound collection direction, and the directional microphone A step of collecting sound from the sound collecting direction and a step of outputting the sound collected by the directional microphone into the vehicle by a speaker are executed. With this configuration, as in the above-described vehicle exterior sound providing device, sounds such as activities and events performed outside the vehicle can be provided inside the vehicle, and the quality of the drive can be improved. In addition, various configurations of the above-described vehicle exterior sound providing apparatus can be applied to the program of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, it has the effect that the sound of the activity, event, etc. currently performed outside a vehicle can be provided in a vehicle, and the quality of a drive can be improved, and it is useful as an apparatus etc. which are mounted in a motor vehicle.

It is a figure which shows the structure of the vehicle exterior sound provision apparatus of 1st Embodiment. It is a figure which shows the example of the data memorize | stored in the road map database. It is a figure which shows the example of the data memorize | stored in the microphone sensitivity table. It is a figure which shows the example which calculates | requires a target area | region based on a person area. It is a figure which shows the hardware constitutions of the vehicle exterior sound provision apparatus of 1st Embodiment. It is a figure which shows operation | movement of the vehicle exterior sound provision apparatus of 1st Embodiment. It is a figure which shows the operation | movement of the sound collection direction determination process by the vehicle exterior sound provision apparatus of 1st Embodiment. It is a figure for demonstrating the method of the determination of the sound collection direction in 2nd Embodiment. It is a figure which shows the sound collection direction determination process of the vehicle exterior sound provision apparatus of 2nd Embodiment. It is a figure which shows the structure of the vehicle exterior sound provision apparatus of 3rd Embodiment. It is a figure which shows the example of the data memorize | stored in the road map database. It is a figure which shows operation | movement of the vehicle exterior sound provision apparatus of 3rd Embodiment. It is a figure which shows operation | movement of the vehicle exterior sound provision apparatus of 4th Embodiment. It is a figure which shows the structure of the vehicle exterior sound provision apparatus of 5th Embodiment. It is a figure which shows operation | movement of the vehicle exterior sound provision apparatus of 5th Embodiment. It is a figure which shows the structure of the vehicle exterior sound provision apparatus of 6th Embodiment. It is a figure which shows operation | movement of the vehicle exterior sound provision apparatus of 6th Embodiment.

Hereinafter, a vehicle exterior sound providing apparatus according to an embodiment of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a diagram illustrating a configuration of a vehicle exterior sound providing apparatus 10 according to the first embodiment. The vehicle exterior sound providing device 10 is a device used by being mounted on an automobile. The vehicle exterior sound providing device 10 acquires an infrared vehicle 11 that captures the vehicle exterior, a directional microphone 12 that collects the vehicle exterior sound, a speaker 13 that outputs the vehicle exterior sound to the vehicle interior, and outputs the vehicle exterior sound to the vehicle interior. Thus, the control unit 14 for controlling the directional microphone 12 and the like, a road map database (hereinafter, “road map DB”) 15, and a microphone sensitivity table 16 are provided.

  FIG. 2 is a diagram illustrating an example of data stored in the road map DB 15. The road map DB 15 stores data representing roads and data representing region classifications. As the region classification, as shown in FIG. 2, classification such as residential area, office area, and downtown area is stored. In FIG. 2, a map is illustrated, but in an actual database, data indicating the location of a region and its classification are stored in association with each other. The classification of these areas may be determined by the provider of this apparatus, or may be automatically determined by analyzing the POI on the map.

  FIG. 3 is a diagram illustrating an example of data stored in the microphone sensitivity table 16. The microphone sensitivity table 16 stores data indicating the region classification and microphone sensitivity in association with each other. This table may be determined by the provider of the apparatus. In the example shown in FIG. 3, it is set to −36 db in a residential area and −20 db in a downtown area. Therefore, the sensitivity of the microphone is increased in the downtown area, and the sensitivity of the microphone is decreased in the residential area. Thereby, for example, in a residential area, it is possible to prevent collecting voices of people in the house and to protect privacy. Although the sensitivity is set to be low in a residential area here, the sensitivity may be set to “0”, that is, sound collection may not be performed.

  The control unit 14 has functions of a road area detection unit 17, a person detection unit 18, a sound collection direction determination unit 19, a directional microphone control unit 20, a position data acquisition unit 21, a vehicle exterior sound output unit 22, and a vehicle speed data acquisition unit 23. Have. The road area detection unit 17 detects a road area from the image captured by the infrared camera 11. Since an approximate region in which an image of a road appears in the image is known from the mounting direction of the infrared camera 11, the road region can be detected by obtaining a boundary line between the road and a region other than the road by edge detection or the like.

  The person detection unit 18 detects a person from the image by detecting a heat source having a predetermined size or more from the image captured by the infrared camera 11. Here, the detected area is referred to as a “person area”. The predetermined size is, for example, a height of 150 cm when converted into an actual coordinate system. Thus, a person can be accurately detected by detecting a heat source having a predetermined size or more.

  The sound collection direction determination unit 19 obtains a target region to be collected based on the person region existing in the region excluding the road obtained by the road region detection unit 17, and determines the direction of the target region as the sound collection direction. .

  FIG. 4 is a diagram illustrating an example of obtaining the target area based on the person area. The sound collection direction determination unit 19 determines a predetermined area surrounding the person area as the target area. Here, the predetermined range may be an ellipse as shown in FIG. 4 or a rectangle. When a target area overlaps with an adjacent person area, a plurality of overlapping target areas are combined and handled as one target area.

  The directional microphone control unit 20 controls the sound collection direction and sensitivity of the directional microphone 12. The directional microphone control unit 20 directs the sound collection direction of the microphone in the direction determined by the sound collection direction determination unit 19 described above. The control unit 14 acquires data on the current position of the vehicle from the position detection unit 30 by using the position data acquisition unit 21. The directional microphone control unit 20 reads out region classification data corresponding to the acquired position data from the road map DB 15, and reads out microphone sensitivity data corresponding to the read out region classification from the microphone sensitivity table 16. The directional microphone control unit 20 controls the sensitivity of the directional microphone 12 so that the read microphone sensitivity is obtained. Further, the directional microphone control unit 20 may control the directivity angle of the directional microphone 12 according to the size of the target region and the distance to the target region.

  The vehicle exterior sound output unit 22 has a function of outputting the vehicle exterior sound collected by the directional microphone 12 from the speaker 13. The vehicle speed data acquisition unit 23 acquires data on the traveling speed of the vehicle from a speed sensor 31 attached to the vehicle. The control unit 14 performs control so as not to provide vehicle exterior sound when the vehicle speed exceeds a predetermined threshold.

  FIG. 5 is a diagram illustrating a hardware configuration of the vehicle exterior sound providing apparatus 10 illustrated in FIG. 1. The vehicle exterior sound providing apparatus 10 includes a CPU 40, a memory 41, an infrared camera 42, a directional microphone 43, a hard disk 44, a speaker 45, and a communication interface 46. Connected by. The memory 41 stores a program 48 for realizing the function of the control unit 14 described with reference to FIG. When the CPU 40 reads and executes the program 48 stored in the memory 41, each function of the control unit 14 shown in FIG. 1 is realized, and operations described later are executed. Such a program 48 is also included in the scope of the present invention.

  The hard disk 44 stores the road map DB 15 and the microphone sensitivity table 16 shown in FIG. The communication interface 46 is an interface for communicating with various sensors and other devices mounted on the vehicle. For example, the communication interface 46 receives current position data from the position detection unit 30 or receives vehicle speed data from the speed sensor 31.

  FIG. 6 is a diagram illustrating an operation of the vehicle exterior sound providing apparatus 10 according to the first embodiment. The vehicle exterior sound providing apparatus 10 repeatedly executes the flow shown in FIG. 6 to provide the interior of the vehicle with sounds heard by the person, the conversation of the person, and the like when a person is detected outside the vehicle.

  The vehicle exterior sound providing apparatus 10 first receives vehicle speed data from the speed sensor 31 and determines whether or not the vehicle speed has become equal to or lower than a predetermined threshold (S10). If the vehicle speed is not less than or equal to the predetermined threshold (NO in S10), this process is repeated until the vehicle speed reaches the predetermined threshold.

  If the vehicle speed is equal to or lower than the predetermined threshold value (YES in S10), a process of providing outside sound to the vehicle is started. First, the vehicle exterior sound providing apparatus 10 captures the periphery of the vehicle with the infrared camera 11 (S12). The vehicle exterior sound providing apparatus 10 performs a heat source region detection process having a certain size or more in the captured image (S14), and determines whether or not such a heat source has been detected (S16). The vehicle exterior sound providing apparatus 10 treats a heat source above a certain level detected here as a person. That is, the area of the heat source detected in the image is the “person area”.

  When the heat source with a certain level or larger is not detected (NO in S16), the vehicle exterior sound providing apparatus 10 returns to the process of comparing the vehicle speed with a predetermined threshold again (S10). When a heat source with a certain level or more is detected (YES in S16), the vehicle exterior sound providing apparatus 10 performs a sound collection direction determination process (S18).

  FIG. 7 is a diagram illustrating the operation of the sound collection direction determination process. The vehicle exterior sound providing apparatus 10 detects a road area from an image photographed by the infrared camera 11 (S30). Next, it is determined whether or not the person area exists in an area other than the road area (S32). If the person area exists in an area other than the road area (YES in S32), the target area is obtained based on the person area. Here, a predetermined area surrounding the person area is obtained as a target area and added (S34). When adjacent target areas overlap, they are handled as one target area. When the person area exists in the road area (NO in S32), it is determined whether or not the next person area exists in the road area without obtaining the target area based on the person area ( S32). The above processing (S32, S34) is repeated for all person areas.

  After performing the process of obtaining the target area based on all person areas, the vehicle exterior sound providing apparatus 10 determines the sound collection direction based on the target area (S36). Specifically, the vehicle exterior sound providing apparatus 10 determines the sound collection direction so that the target area is included in the sound collection range.

  With reference to FIG. 6, the operation of the vehicle exterior sound providing apparatus 10 after the sound collection direction is determined will be described. The vehicle exterior sound providing apparatus 10 receives data on the current position of the vehicle from the position detection unit 30 and reads out region classification data on the current position from the road map DB 15. Subsequently, the vehicle exterior sound providing apparatus 10 reads the microphone sensitivity data corresponding to the area classification data from the microphone sensitivity table 16, and determines the sensitivity of the directional microphone 12 (S20). With the processing so far, the sound collection direction and sensitivity of the directional microphone 12 are determined. Next, the directional microphone control unit 20 of the vehicle exterior sound providing apparatus 10 directs the directional microphone 12 in the determined sound collection direction and collects sound with the determined microphone sensitivity (S22). Subsequently, the vehicle exterior sound providing apparatus 10 outputs the sound collected by the directional microphone 12 from the speaker 13 into the vehicle (S24).

  The configuration and operation of the vehicle exterior sound providing apparatus 10 according to the first embodiment of the present invention have been described above. The vehicle exterior sound providing apparatus 10 according to the first embodiment obtains a target area based on a person area in an area other than a road, and determines a sound collection direction so that sound can be collected from the target area. As a result, it is possible to provide the inside of the vehicle with sounds heard by a person in an area other than the road and the conversation of the person, and a sense of unity between the outside and the inside of the vehicle can be realized. In addition, since sound is collected from a region other than the road, it is difficult to pick up harsh sounds from other vehicles on the road.

  In addition, since the vehicle exterior sound providing apparatus 10 according to the first embodiment does not provide vehicle exterior sound when the vehicle speed is equal to or higher than a predetermined threshold value, an environment that can concentrate on driving can be created when the vehicle speed is high. it can.

  In addition, the vehicle exterior sound providing apparatus 10 according to the first embodiment collects sound with the directional microphone 12 with sensitivity corresponding to the current position of the vehicle. For example, the sensitivity of the microphone is reduced in a quiet residential area. It can be controlled so that it does not collect even private conversations, and it can be collected in a privacy-friendly manner.

  In the present embodiment, the example in which the sensitivity of the directional microphone 12 is changed according to the region classification has been described. However, whether or not to provide outside sound may be changed according to the type of road. In this case, a table is prepared in which the type of road is associated with whether or not external sound can be provided. The road type on which the vehicle is traveling is obtained from the current position acquired by the position data acquisition unit 21, and data on whether or not to provide outside sound corresponding to the road type is read from the table. The vehicle exterior sound providing apparatus 10 can determine whether or not the vehicle exterior sound can be provided based on the data read from the table. For example, when the road type is an expressway, processing such as not providing outside sound can be performed.

  In the present embodiment, the example in which the sensitivity is changed according to the region classification has been described. However, the sensitivity may be changed according to the number of persons included in the target area. For example, when the number of people in the target area is a predetermined number (for example, three people) or less, there is a possibility of a private conversation, so the sensitivity of the directional microphone 12 may be lowered.

  In the present embodiment, the target area may be obtained based on the person area closest to the vehicle. Thereby, the utterance content of the person who is talking to the own vehicle can be transmitted to the inside of the vehicle.

  Also, not all person areas in areas other than roads are targeted areas. For example, when a person in a person area is moving or alone, it is based on such person area. The target area may not be obtained. There is a possibility that a moving person or a single person is not talking, and the process of collecting sound using the directional microphone 12 may be wasted. The possibility of acquiring sound outside the vehicle more efficiently is more appropriate for collecting sound.

(Second Embodiment)
Next, the vehicle exterior sound providing apparatus 10 according to the second embodiment of the present invention will be described. The basic configuration of the vehicle exterior sound providing apparatus 10 of the second embodiment is the same as that of the first embodiment (see FIG. 1), but the vehicle exterior sound providing apparatus 10 of the second embodiment is The method of determining the sound direction is different from that of the first embodiment.

  FIG. 8 is a diagram for explaining how to determine the sound collection direction in the second embodiment. In the example shown in FIG. 8, a person is detected at four locations A1 to A4 around the vehicle. These four person areas all exist in areas other than the road. In the second embodiment, the vehicle exterior sound providing apparatus 10 does not obtain a target area based on a person area on the opposite side across the opposite lane among person areas existing in an area other than the road. In the example shown in FIG. 8, the target area based on the person areas A3 and A4 is not obtained, but the target area based on the person areas A1 and A2 is obtained. And the vehicle exterior sound provision apparatus 10 determines the direction of a target area | region as a sound collection direction.

  FIG. 9 is a diagram illustrating a sound collection direction determination process of the vehicle exterior sound providing apparatus 10 according to the second embodiment. The vehicle exterior sound providing apparatus 10 detects a road area from an image photographed by the infrared camera 11 (S30). Next, it is determined whether or not the person area exists in an area other than the road area (S32). If the person area exists in an area other than the road area (YES in S32), it is determined whether or not the person area is on the opposite side across the opposite lane (S33). If it is determined that the vehicle is not on the opposite side across the opposite lane (NO in S33), a target area based on the person area is added (S34). If the person area exists in the road area (NO in S32) or if the person area is on the opposite side across the opposite lane (YES in S33), the target area based on the person area is not obtained and the next It is determined whether the person area is present in the road area (S32). The above processing (S32, S33, S34) is repeated for all person areas.

  After performing the process of obtaining the target area based on all person areas, the vehicle exterior sound providing apparatus 10 determines the sound collection direction based on the target area (S36). Specifically, the vehicle exterior sound providing apparatus 10 determines the sound collection direction so that the target area is included in the sound collection range.

  Since the vehicle exterior sound providing apparatus 10 according to the second embodiment excludes the region on the side across the opposite lane from the sound collection direction, it is possible to collect the sound of the oncoming vehicle when collecting sound from the target region. In addition, noise collection noise can be reduced.

(Third embodiment)
Next, a vehicle exterior sound providing apparatus 10a according to a third embodiment of the present invention will be described. Although the basic configuration of the vehicle exterior sound providing apparatus 10a of the third embodiment is the same as that of the first embodiment (see FIG. 1), the vehicle exterior sound providing apparatus 10a of the third embodiment is The method of determining the sound direction is different from that of the first embodiment.

  FIG. 10 is a diagram illustrating a configuration of the vehicle exterior sound providing apparatus 10a according to the third embodiment. The basic configuration of the vehicle exterior sound providing apparatus 10a of the third embodiment is the same as that of the first embodiment, but the data stored in the road map DB 15a is different, and the sound collection direction determination unit 19a Processing is different. The road map DB 15a stores POI data in addition to the regional classification data described in the first embodiment.

  FIG. 11 is a diagram illustrating an example of POI data stored in the road map DB 15a. In FIG. 11, a map including POI is illustrated, but in an actual database, data indicating the position of POI, POI attributes, and the like are stored in association with each other. By this road map DB 15a, it is possible to grasp what facilities are located at which places.

  FIG. 12 is a diagram illustrating a sound collection direction determination process in the third embodiment. The vehicle exterior sound providing apparatus 10a detects a road area from an image photographed by the infrared camera 11 (S30). Next, it is determined whether or not the person area exists in an area other than the road area (S32). If the person area exists in an area other than the road area (YES in S32), it is determined whether or not the person area is within a predetermined range from the facility (S33a). If it is determined that the person area is within the predetermined range (YES in S33a), a target area based on the person area is added (S34). If the person area exists in the road area (NO in S32) or if the person area is not within the predetermined range of the facility (NO in S33a), the next person is determined without obtaining the target area based on the person area. A process of determining whether the area exists within the road area is performed (S32). The above processing (S32, S33a, S34) is repeated for all person areas.

  After performing the process of obtaining the target area based on all person areas, the vehicle exterior sound providing apparatus 10a determines the sound collection direction based on the target area (S36). Specifically, the vehicle exterior sound providing apparatus 10a determines the sound collection direction so that the target area is included in the sound collection range.

  The vehicle exterior sound providing apparatus 10a according to the third embodiment determines the sound collection direction using a person area within a predetermined range from the facility as a target area, so that the possibility of acquiring sound related to an event or activity is increased. Can improve the quality.

  In this embodiment, the example of using the facility data stored in the road map DB 15 as it is has been described. However, the attribute of the facility for collecting sound may be specified according to the passenger's preference or the like. For example, in the case of an occupant who likes to eat and walk, by specifying restaurant attributes, sound can be collected from a person area around the restaurant, and voice that meets the occupant's preference can be acquired.

(Fourth embodiment)
Next, the vehicle exterior sound providing apparatus 10 according to the fourth embodiment will be described. The basic configuration of the vehicle exterior sound providing apparatus 10 of the fourth embodiment is the same as that of the first embodiment. In the above-described embodiment, sound collection is performed using a person area as a target area. However, the fourth embodiment is different in that sound collection is performed using an area other than the person area as a target area. As a result, it is possible to enjoy the sounds of nature by the road (river stream, sea sound, insect voice, etc.) without being disturbed by human voice.

  FIG. 13 is a diagram illustrating sound collection direction determination processing by the vehicle exterior sound providing apparatus 10 according to the fourth embodiment. The vehicle exterior sound providing apparatus 10 detects a road area from an image photographed by the infrared camera 11 (S40). Next, the vehicle exterior sound providing apparatus 10 adds the direction of the person area as a direction in which sound is not collected (this is referred to as “non-sound collection direction”) (S42). The non-sound collection direction is obtained for all person areas. Next, the vehicle exterior sound providing apparatus 10 excludes the non-sound collecting direction from the direction in which the region other than the road region exists, and determines the remaining direction as the sound collecting direction (S44).

(Fifth embodiment)
FIG. 14 is a diagram illustrating a configuration of the vehicle exterior sound providing apparatus 10b according to the fifth embodiment of the present invention. In addition to the configuration of the vehicle exterior sound providing apparatus 10 according to the first embodiment, the vehicle exterior sound providing apparatus 10b according to the fifth embodiment includes a provision timing determining unit 25 for determining the timing for providing outside sound, and a sound. A sound data storage unit 26 for storing data is provided. The vehicle exterior sound providing apparatus 10b according to the fifth embodiment includes a vehicle state data acquisition unit 24 that receives not only vehicle speed data but also shift position data. The vehicle state data acquisition unit 24 is connected to a speed sensor 31 and a shift sensor 32 attached to the vehicle.

  The vehicle exterior sound providing apparatus 10 b according to the fifth embodiment stores the vehicle exterior sound acquired by the directional microphone 12 in the sound data storage unit 26. The provision timing determination unit 25 determines whether or not the vehicle exterior sound may be provided based on the vehicle state acquired by the vehicle state data acquisition unit 24, and determines the timing at which the vehicle may be provided. The vehicle exterior sound output unit 22 reads the sound data from the sound data storage unit 26 and outputs the sound data from the speaker 13 when the provision timing determination unit 25 determines that the vehicle exterior sound may be provided.

  FIG. 15 is a diagram illustrating the operation of the vehicle exterior sound providing apparatus 10b according to the fifth embodiment. The processing up to the processing for collecting the vehicle outside sound by the vehicle outside sound providing apparatus 10b of the fifth embodiment (S10 to S22) is the same as that of the first embodiment. The vehicle exterior sound providing apparatus 10b according to the fifth embodiment temporarily stores the collected vehicle exterior sound in the sound data storage unit 26 (S50).

  Next, the vehicle exterior sound providing apparatus 10b acquires vehicle speed data and shift position data at the vehicle state data acquisition unit 24, and determines whether vehicle exterior sound can be provided based on the vehicle state. (S52). Specifically, when the vehicle is accelerating / decelerating, or when the shift position is “reverse (back)”, it is determined that the outside sound cannot be provided. If it is determined that the vehicle exterior sound cannot be provided (NO in S52), the process returns to the process of determining the vehicle speed and the predetermined threshold (S10), and the vehicle exterior sound is continuously collected.

  If it is determined that the vehicle exterior sound can be provided (YES in S52), the vehicle exterior sound providing apparatus 10b reads the vehicle exterior sound data from the sound data storage unit 26, and converts the read data into an analog signal. (S54), a sound outside the vehicle is output from the speaker 13 (S56).

  According to the fifth embodiment, since it is determined whether or not the vehicle exterior sound may be provided according to the state of the vehicle, the vehicle exterior sound can be prevented from being output at a timing that hinders driving. , Can ensure safety.

  In the above-described embodiment, the vehicle exterior sound collected when the vehicle exterior sound cannot be provided may be discarded or provided when it is time to provide the vehicle exterior sound. When the vehicle exterior sound is provided later at the time when it can be provided, the delayed time may be shortened by cutting the silence period included in the collected vehicle exterior sound.

  In the above-described embodiment, it is determined whether the vehicle is accelerating / decelerating or backing as the vehicle state. However, this is an example, and various other vehicle states are used for determining whether or not to provide the vehicle state. Can do. For example, if it is detected that the vehicle is going to turn left or right or that the vehicle is going to change lanes based on the state of the blinker, the outside sound may not be provided. Moreover, when it is judged that it is raining based on the state of the wiper, the sound outside the vehicle may not be provided. Further, based on the lighting state of the light, when the light is on, it may be determined that the surroundings are dark and the outside sound may not be provided.

(Sixth embodiment)
FIG. 16 is a diagram illustrating a configuration of the vehicle exterior sound providing apparatus 10 according to the sixth embodiment. Although the basic configuration of the vehicle exterior sound providing apparatus 10 of the sixth embodiment is the same as that of the first embodiment, the sound collected by the directional microphone 12 is stored in the sixth embodiment. A sound data storage unit 26 that performs sound output, and an output sound determination unit 27 that determines which vehicle exterior sound is to be output among the vehicle exterior sounds acquired from a plurality of directions. The output sound determination unit 27 determines the vehicle exterior sound having the highest volume as the vehicle exterior sound to be output from the plurality of vehicle exterior sounds acquired in the same period.

  FIG. 17 is a diagram illustrating the operation of the vehicle exterior sound providing apparatus 10 according to the sixth embodiment. The processing until the vehicle exterior sound providing apparatus 10 according to the sixth embodiment collects the vehicle exterior sound (S10 to S22) is the same as that of the first embodiment. The vehicle exterior sound providing apparatus 10 according to the sixth embodiment temporarily stores the collected vehicle exterior sound in the sound data storage unit 26 (S50).

  The vehicle exterior sound providing apparatus 10 according to the sixth embodiment determines whether there is sound data from a plurality of sound collection directions in the sound data storage unit 26 (S60). If there is no sound data from a plurality of sound collection directions (NO in S60), the data of the sound outside the vehicle is read from the sound data storage unit 26, the read data is converted into an analog signal (S64), and the outside of the vehicle from the speaker 13 A sound is output (S66).

  When there is sound data from a plurality of sound collection directions (YES in S60), the vehicle exterior sound providing apparatus 10 determines from which direction the sound collected is output based on the sound volume (S62). . Then, the sound data collected from the determined direction is read from the sound data storage unit 26, the read data is converted into an analog signal (S64), and the vehicle exterior sound is output from the speaker 13 (S66).

  Since the vehicle exterior sound providing apparatus 10 of the sixth embodiment provides the vehicle exterior sound having the maximum volume among the vehicle exterior sounds collected from a plurality of directions, priority is given to a sound that is highly likely to be valuable to the occupant. Can be provided.

  In addition, although the vehicle exterior sound provision apparatus 10 of 6th Embodiment demonstrated the example which selects a sound based on a sound volume, it is good also as selecting a sound based on a frequency. For example, sound data including the most frequency component of human voice may be selected and output.

  As mentioned above, although the embodiment has been described in detail for the configuration of the vehicle exterior sound providing apparatus 10 of the present invention, the present invention is not limited to the above-described embodiment.

  In the above-described embodiment, the infrared camera 11 is used. However, the camera used in the present invention is not necessarily the infrared camera 11, and a normal camera may be used.

  In the sixth embodiment described above, an example in which the vehicle exterior sound to be output when sound collection is performed from a plurality of directions has been described. However, all vehicle exterior sounds collected by a plurality of directional microphones are all described. It may be provided. In this case, using the surrounding system, each collected sound is output from the collected direction. Thereby, the sound heard outside a vehicle can be reproduced appropriately.

  In the above-described embodiment, a sign indicating that sound outside the vehicle is being collected may be displayed while sound outside the vehicle is being collected. By attracting attention to people outside the car in this way, it is possible to collect only conversations that only enter the ears when people are walking normally, and not to collect privacy-related conversations. can do.

  INDUSTRIAL APPLICABILITY The present invention provides effects such as providing sounds inside activities and events that are performed outside the vehicle and improving the quality of the drive, and is useful as a device that is mounted on a vehicle and used.

DESCRIPTION OF SYMBOLS 10 Outside vehicle sound provision apparatus 11 Infrared camera 12 Directional microphone 13 Speaker 14 Control part 15, 15a Road map database 16 Microphone sensitivity table 17 Road area detection part 18 Person detection part 19, 19a Sound collection direction determination part 20 Directional microphone control part 21 position data acquisition unit 22 vehicle exterior sound output unit 23 vehicle speed data acquisition unit 24 vehicle state data acquisition unit 25 provision timing determination unit 26 sound data storage unit 27 output sound determination unit 30 position detection unit 31 speed sensor 32 shift sensor

Claims (21)

A camera that shoots outside the car,
A person detection unit for detecting a person outside the vehicle;
A directional microphone that collects sounds outside the vehicle,
An area other than the road is detected from the image captured by the camera, a target area to be collected is obtained from the area other than the road based on the detection result by the person detection unit, and the direction of the target area is collected. A sound collection direction determination unit that determines the sound direction;
A directional microphone controller that directs the directional microphone in the sound collection direction;
A speaker that outputs the sound collected by the directional microphone into the vehicle;
A vehicle exterior sound providing device.   The vehicle exterior sound providing apparatus according to claim 1, wherein the camera is an infrared camera, and the person detection unit detects a heat source having a predetermined size or more as a person from an image captured by the infrared camera. A vehicle speed data acquisition unit that acquires vehicle speed data from a speed sensor that detects a vehicle speed;
The vehicle exterior sound providing apparatus according to claim 1 or 2, wherein the vehicle exterior sound is not provided when the vehicle speed is equal to or higher than a predetermined threshold.   The vehicle exterior sound providing apparatus according to any one of claims 1 to 3, wherein the sound collection direction determination unit obtains the target region including a person region detected by the person detection unit in a region other than a road.   The vehicle exterior sound providing apparatus according to claim 4, wherein the sound collection direction determination unit does not include the person area in the target area when the detected person is moving.   The vehicle exterior sound providing apparatus according to claim 4, wherein the sound collection direction determination unit does not include the person area in the target area when the detected person is alone. A position data acquisition unit for acquiring data on the current position of the vehicle;
A road map database storing road maps including facility data;
With
The sound collection direction determination unit determines whether the person area is within a predetermined range including facilities based on the current position of the vehicle acquired by the position data acquisition unit and the data stored in the road map database. The vehicle exterior sound providing apparatus according to claim 4, wherein if the person area is not within the predetermined range, the person area is not included in the target area.   The vehicle exterior sound providing apparatus according to claim 4, wherein the sound collection direction determination unit obtains a target area based on a person area within a predetermined distance from the vehicle.   The outside sound providing apparatus according to any one of claims 4 to 8, wherein the directional microphone control unit lowers the sensitivity of the directional microphone when the number of persons in the target area is equal to or less than a predetermined threshold value. A position data acquisition unit for acquiring data on the current position of the vehicle;
A road map database storing road maps including data indicating the classification of regions;
A microphone sensitivity table that stores the classification of the region and the sensitivity of the directional microphone in association with each other;
With
The directional microphone control unit reads out the classification of the region corresponding to the current position of the vehicle acquired by the position data acquisition unit from the road map database, and sets the sensitivity of the directional microphone corresponding to the read out classification of the region. The vehicle exterior sound providing apparatus according to any one of claims 1 to 9, wherein the directional microphone is controlled to read out from a microphone sensitivity table and collect sound with the read sensitivity. A position data acquisition unit for acquiring data on the current position of the vehicle;
A road map database storing road maps including data indicating road types;
A provision availability table in which data on whether or not to provide road type and external sound is stored in association with each other;
With
The road type corresponding to the current position of the vehicle acquired by the position data acquisition unit is read from the road map database, and based on the read road type and the data stored in the provision availability table, The vehicle exterior sound providing apparatus according to any one of claims 1 to 10, which determines whether or not to provide the sound.   The vehicle exterior sound providing apparatus according to any one of claims 1 to 11, wherein the sound collection direction determination unit does not include a person area in an area opposite to the host vehicle with the opposite lane in between.   The vehicle exterior sound providing apparatus according to any one of claims 1 to 3, wherein the sound collection direction determination unit obtains a direction in which a person is not detected by the person detection unit in a region other than a road as a sound collection direction. An input unit for receiving selection of whether the direction in which the person is present is the sound collection direction or the direction in which no person is the sound collection direction;
The vehicle exterior sound providing apparatus according to any one of claims 1 to 3, wherein the sound collection direction determination unit determines a direction in which a person is detected or a direction in which a person is not detected as a sound collection direction according to the selection. . A vehicle state data acquisition unit for acquiring data indicating the state of the vehicle;
A provision timing determining unit that determines a timing for outputting a vehicle exterior sound from the speaker based on the vehicle state data acquired by the vehicle state data acquiring unit;
The vehicle exterior sound provision apparatus in any one of Claims 1-14 provided with these. The vehicle state data acquisition unit acquires vehicle speed data from a speed sensor that detects the speed of the vehicle,
The vehicle exterior sound providing apparatus according to claim 15, wherein the provision timing determining unit determines that the vehicle exterior sound is not output when the vehicle is accelerating or decelerating. The vehicle state data acquisition unit acquires data indicating a shift position from a shift sensor that detects the shift position,
The vehicle exterior sound providing apparatus according to claim 15, wherein the provision timing determining unit determines that the vehicle exterior sound is not output when the shift is in a “reverse” position.   The vehicle exterior sound provision according to any one of claims 1 to 17, wherein when the directional microphone collects sound from a plurality of sound collection directions, a sound to be provided in the vehicle is selected based on the volume of the collected sound. apparatus.   The vehicle exterior sound providing device according to any one of claims 1 to 17, wherein when the directional microphone collects sound from a plurality of sound collecting directions, a sound to be provided in the vehicle is selected based on the frequency of the collected sound. . A method for providing sound outside the vehicle to the inside of the vehicle,
A vehicle outside sound providing device acquiring an image outside the vehicle photographed by a camera;
The vehicle exterior sound providing device acquires data of a person outside the vehicle detected by a person detection unit;
The vehicle exterior sound providing device detects an area other than a road from an image captured by the camera, and obtains a target area to be collected from an area other than the road based on a detection result by the person detection unit. Determining the direction of the target area as the sound collection direction;
The vehicle outside sound providing device collecting sound from the sound collecting direction with a directional microphone;
The vehicle exterior sound providing device outputs a sound collected by a directional microphone into a vehicle with a speaker;
A vehicle exterior sound providing method. A program for providing sound outside the vehicle, to a computer connected to a camera that captures the outside of the vehicle, a directional microphone that collects sound outside the vehicle, and a speaker that outputs sound inside the vehicle,
Obtaining an image outside the vehicle photographed by the camera;
Detecting a person outside the vehicle from the outside image;
An area other than a road is detected from an image captured by the camera, a target area to be collected is obtained from the area other than the road based on the detection result of the person, and the direction of the target area is collected. Determining a direction,
Collecting from the sound collection direction with the directional microphone;
Outputting the sound collected by the directional microphone into the vehicle with a speaker;
A program that executes