JP2014106617A - On-vehicle information providing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an on-vehicle information providing device capable of providing information on a situation around a vehicle.SOLUTION: An on-vehicle information providing device includes: a subject extraction part 11 which extracts an image of the main subject from image data around a vehicle photographed by an on-vehicle camera 10; a built-in database 18 in which attribute information on the subject is stored for each type of the subject; a subject specification part 12 in which, based on the type of the subject discriminated from an image of the subject extracted by the subject extraction part 11, attribute information on the subject is specified from the attribute information stored in the built-in database 18, and appearance information indicating appearance during photographing of the subject is specified from the image of the subject; and a content creation part 15 in which, based on the attribute information and appearance information of the subject specified by the subject specification part 12, content including attribute and appearance of the subject is created.

Description

  The present invention relates to an in-vehicle information providing apparatus that provides information obtained by image analysis of a vehicle periphery image.

  For example, in Patent Document 1, signboard information is extracted from image data obtained by photographing the periphery of a vehicle, and the position of a facility that provides the signboard is determined from the extracted signboard information and the vehicle position where the signboard is photographed. A navigation device for detecting is disclosed. By doing in this way, it becomes possible to obtain the latest information at the time of traveling of the facility on the traveling route of the own vehicle.

JP 2007-078632 A

  The situation around the vehicle generally changes from moment to moment, and even if a picture is taken at the same place, the same image is not always obtained. For example, it is rare if the pedestrians or vehicles photographed at the same position around the vehicle are the same each time, and even the same subject can be photographed in completely different situations depending on the time zone or season.

  In this way, the image data around the vehicle has a rare value that the same image cannot be obtained many times. By using the vehicle periphery image, information indicating various situations around the vehicle that change every moment can be obtained. It is possible to provide. In addition, by processing such information into a game or the like, it is possible to provide entertainment that allows passengers to enjoy the situation around the vehicle.

  Note that the conventional technique represented by Patent Document 1 uses the image data obtained by photographing the periphery of the vehicle within the general range of the navigation function associated with the position information. Therefore, more advanced use of the vehicle periphery image as described above is impossible.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an in-vehicle information providing apparatus capable of providing information on the situation around the vehicle that changes every moment.

  An in-vehicle information providing device according to the present invention inputs image data around a vehicle photographed by an in-vehicle camera, extracts a main subject image from the image data, and attributes related to the subject for each type of subject. The type of the subject is determined from the storage unit that stores the information and the subject image extracted by the subject extraction unit, and the attribute information of the subject is identified from the attribute information stored in the storage unit based on the determined type In addition, the subject specifying unit for specifying appearance information indicating the appearance of the subject at the time of shooting from the subject image, and the subject attribute and appearance information based on the subject attribute information and appearance information specified by the subject specifying unit are included. A content generation unit that generates content, and an information output unit that provides the content generated by the content generation unit to users in the vehicle

  According to the present invention, there is an effect that it is possible to provide information on the situation around the vehicle that changes from moment to moment.

It is a block diagram which shows the structure of the vehicle-mounted information provision apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of a to-be-identified part. It is a flowchart which shows the process which provides the information regarding a vehicle based on vehicle periphery image data. It is a flowchart which shows the process which provides the information regarding a person based on vehicle periphery image data. It is a flowchart which shows the process which provides the information regarding an animal based on vehicle periphery image data. It is a flowchart which shows the process which provides the information regarding an airplane based on vehicle periphery image data. It is a flowchart which shows the process which provides the information regarding a plant based on vehicle periphery image data. It is a flowchart which shows the process which provides the information regarding a label | marker based on vehicle periphery image data. It is a flowchart which shows the process which provides the information regarding a feature based on vehicle periphery image data. It is a figure which shows the outline | summary of the vehicle-mounted information provision system which concerns on Embodiment 2 of this invention.

Embodiment 1 FIG.
1 is a block diagram showing a configuration of an in-vehicle information providing apparatus according to Embodiment 1 of the present invention. As shown in FIG. 1, an in-vehicle information providing apparatus 1 is an in-vehicle information providing apparatus that is mounted on a vehicle and provides information obtained from a vehicle peripheral image. The identification unit 12, the storage unit 13, the auxiliary information acquisition unit 14, the content generation unit 15, the application provision unit 16, and the information output unit 17 are configured.
The in-vehicle camera 10 is an in-vehicle camera that has a field of view around the vehicle, and a rear camera, a front camera, and a side camera that are mounted on the vehicle as a standard may be used.
The subject extraction unit 11 inputs image data around the vehicle captured by the in-vehicle camera 10 while the vehicle is traveling, and extracts an image of the main subject from the image data. For example, various subject extraction algorithms are executed to extract the main part (main subject) in the image.

  The subject specifying unit 12 determines the type of the subject from the subject image extracted by the subject extracting unit 11. For example, by using pattern recognition technology, it is analyzed whether the subject matches a pattern corresponding to a preset type (vehicle, person, animal, aircraft, plant, sign, feature, etc.) Determine the type. Further, the subject specifying unit 12 specifies attribute information of the subject from the attribute information stored in the built-in database 18 by referring to the built-in database 18 described later with reference to FIG. Further, the subject specifying unit 12 specifies appearance information indicating the appearance of the subject at the time of shooting from the subject image extracted by the subject extracting unit 11.

  The storage unit 13 is a storage unit that associates the attribute information, appearance information, and shooting date / time information of the subject specified by the subject specifying unit 12 and stores them in the data storage area 13a as history information of the subject. As the storage unit 13, for example, a hard disk drive (HDD) device, a USB memory, and a storage medium (CD, DVD, BD) that can be played back by a storage media playback device mounted on the in-vehicle information providing device 1 are constructed. When the storage unit 13 is a storage media playback device, a data storage area 13a is constructed in the storage media played back by this device.

  The auxiliary information acquisition unit 14 is an information acquisition unit that acquires attribute information by communicating with an external database that stores attribute information about the subject for each type of subject. For example, when the attribute information is acquired by connecting to a specific external database that stores the attribute information, a search key including the type of the subject is generated, and the attribute is searched by a search engine on the Internet. Also good.

  The content generation unit 15 is a content generation unit that generates content including subject attributes and appearance based on subject attribute information and appearance information identified by the subject identifying unit 12. For example, if the type of subject is a vehicle, content that introduces a vehicle that is different from the subject vehicle is generated, and in addition to the attribute information such as the type and performance of the vehicle, the appearance of the vehicle at the time of shooting is shown. introduce. The appearance may indicate the color of the vehicle, but may also be the exterior or the presence or absence of dirt. Thereby, the user can know that the vehicle has passed a special exterior, and can recognize it as unfortunate information if the appearance of the luxury vehicle is dirty. Such information is so-called momentary information obtained by photographing with the in-vehicle camera 10, and the user can enjoy vehicle surrounding information at that time.

The application providing unit 16 has an application that realizes various information providing methods, and provides an application in response to a request from the content generation unit 15. As an application provided by the application providing unit 16, for example, an information providing method for displaying information in a list format and introducing information around the vehicle may be realized, or provided in a quiz format. You may implement | achieve the interactive information provision method which makes a passenger answer.
Note that the application providing unit 16 may be omitted when only the application preset in the content generating unit 15 is used.

The information output unit 17 outputs the content generated by the content generation unit 15. For example, it includes an in-vehicle display that visually provides content and an in-vehicle speaker that provides audio.
In addition, as described later in the second embodiment, content may be generated by a portable information terminal such as a smartphone, and an in-vehicle audio device with a display may function as the information output unit 17 to provide the content. Good.

  The subject extraction unit 11, the subject identification unit 12, the auxiliary information acquisition unit 14, the content generation unit 15, and the application provision unit 16 are described by the CPU mounted in the in-vehicle information provision device 1 for describing each function of these components By executing the program, it is realized as a processing means in which software and hardware cooperate.

  FIG. 2 is a block diagram showing a configuration of the subject specifying unit, and shows an internal functional configuration of the subject specifying unit 12 of FIG. The subject specifying unit 12 shown in FIG. 2 includes a vehicle specifying unit 12a, a person specifying unit 12b, an animal specifying unit 12c, a flying object specifying unit 12d, a plant specifying unit 12e, a sign specifying unit 12f, a feature specifying unit 12g, and a built-in database 18. It is configured with. In FIG. 2, the subject specifying unit 12 includes all of the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, the sign specifying unit 12f, and the feature specifying unit 12g. However, what is necessary is just to provide at least one of these.

  The vehicle specifying unit 12a determines the type of the subject based on the subject image extracted by the subject extracting unit 11, and specifies the attribute information of the vehicle with reference to the built-in database 18 when the type is a vehicle. The appearance information at the time of shooting of the vehicle is specified. For example, image patterns of various vehicles are set in advance, and it is determined by this pattern recognition whether or not the subject is a vehicle. Further, the vehicle identification unit 12 a refers to the built-in database 18 using the vehicle image as a search key, and identifies the attribute information of the subject vehicle from the attribute information stored in the built-in database 18. Furthermore, items to be extracted from the image of the subject are set in advance in the vehicle identification unit 12a, and the appearance at the time of shooting is determined along the items to generate appearance information.

  The person specifying unit 12b determines the type of the subject based on the image of the subject extracted by the subject extracting unit 11, and specifies the attribute information of the person with reference to the built-in database 18 when the type is a person. Appearance information at the time of photographing the person is specified. For example, a human image pattern is set in advance, and it is determined by this pattern recognition whether or not the subject is a person. The person specifying unit 12b refers to the built-in database 18 using a person image as a search key, and specifies attribute information of the subject person from the attribute information stored in the built-in database 18. Further, items to be extracted from the subject image are set in advance in the person specifying unit 12b as appearances, and appearances at the time of shooting are determined along the items to generate appearance information.

  The animal specifying unit 12c determines the type of the subject based on the image of the subject extracted by the subject extracting unit 11, and specifies the attribute information of the animal with reference to the built-in database 18 when the type is an animal. Identify appearance information at the time of photographing the animal. For example, image patterns of various animals (animals moving on the ground) are set in advance, and it is determined whether or not the subject is an animal by this pattern recognition. The animal specifying unit 12c refers to the built-in database 18 using an animal image as a search key, and specifies attribute information of the subject animal from the attribute information stored in the built-in database 18. Furthermore, items to be extracted from the subject image are set in advance in the animal specifying unit 12c as appearances, and appearances at the time of photographing are determined along the items to generate appearance information.

  The flying object specifying unit 12d determines the type of the subject based on the image of the subject extracted by the subject extracting unit 11, and refers to the built-in database 18 for attribute information of the flying object when the type is a flying object. Identify the appearance information when shooting the flying object. For example, image patterns of various flying objects (aircraft, birds, etc.) are set in advance, and it is determined by this pattern recognition whether the subject is a flying object. The flying object specifying unit 12d refers to the built-in database 18 using the image of the flying object as a search key, and specifies the attribute information of the flying object of the subject from the attribute information stored in the built-in database 18. Further, items to be extracted from the image of the subject are set in advance in the flying object specifying unit 12d, and the appearance at the time of shooting is determined along the items to generate appearance information.

  The plant specifying unit 12e determines the type of the subject based on the image of the subject extracted by the subject extracting unit 11, and specifies the attribute information of the plant with reference to the built-in database 18 when the type is a plant. Appearance information at the time of photographing the plant is specified. For example, image patterns of various plants (roadside trees such as street trees and flowerbed flowers installed along the roadside) are set in advance, and it is determined whether or not the subject is a plant by this pattern recognition. . The plant specifying unit 12e refers to the built-in database 18 using a plant image as a search key, and specifies the attribute information of the subject plant from the attribute information stored in the built-in database 18. Furthermore, items to be extracted from the image of the subject are set in advance in the plant specifying unit 12e, and the appearance at the time of shooting is determined along the items to generate appearance information.

  The sign specifying unit 12f determines the type of the subject based on the image of the subject extracted by the subject extracting unit 11, and specifies the attribute information of the sign with reference to the built-in database 18 when the type is a sign. Appearance information at the time of photographing the sign is specified. For example, image patterns of various signs are set in advance, and it is determined by this pattern recognition whether or not the subject is a sign. In addition, the sign identifying unit 12f refers to the built-in database 18 using the sign image as a search key, and identifies the attribute information of the object sign from the attribute information stored in the built-in database 18. Furthermore, items to be extracted from the image of the subject are set in advance in the sign specifying unit 12f, and the appearance at the time of shooting is determined along the items to generate appearance information.

  The feature specifying unit 12g determines the type of the subject based on the image of the subject extracted by the subject extraction unit 11, and when the type is the feature, refers to the built-in database 18 to obtain attribute information of the feature. Identify the appearance information at the time of shooting the feature. For example, image patterns of various features (buildings, mountains, rivers, etc. other than roadside installations such as signs) are set in advance, and it is determined by this pattern recognition whether the subject is a feature. The feature identifying unit 12g refers to the built-in database 18 using the feature image as a search key, and identifies the attribute information of the subject's feature from the attribute information stored in the built-in database 18. Further, items to be extracted from the subject image are set in advance in the feature specifying unit 12g, and the appearance at the time of shooting is determined along the items to generate appearance information.

The built-in database 18 is a database that stores attribute information regarding the subject for each type of subject. For example, the pattern image for each subject type and the attribute information of the subject are registered in association with each other. When specifying attribute information, the feature information extracted from the image data of the subject (such as the appearance of the subject) is compared with the pattern image to determine the best match, and based on this, the attribute information of the subject is specified. .
As the built-in database 18, as with the storage unit 13, for example, a storage medium (CD, DVD) that can be played back by a hard disk drive (HDD) device, a USB memory, or a storage media playback device mounted on the in-vehicle information providing device 1. , BD).

Next, the operation will be described.
In the following, processing from specification of a type to generation of content will be described, taking as an example the case where the subject is a vehicle, a person, an animal, a flying object, a plant, a sign, and a feature.
FIG. 3 is a flowchart showing a process for providing information related to the vehicle based on the vehicle periphery image data, and shows the process when the subject is a vehicle.
When the subject specifying unit 12 inputs the image data of the subject extracted by the subject extracting unit 11, the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, and the sign specifying unit The image data is sorted and output to 12f and the feature specifying unit 12g. Thereby, each specification part 12a-12g analyzes a to-be-photographed object, and discriminate | determines whether it is the classification which should identify itself. Here, since the subject is a vehicle, the vehicle specifying unit 12a determines that the type of the subject is a vehicle (step ST1). If the subject is not the type that the subject should identify, the identifying unit stops the subsequent processing.

Next, the vehicle specifying unit 12a determines whether or not the image data of the subject is data that can extract feature points necessary for specifying the attributes and appearance of the vehicle (step ST2).
Here, if the feature point is image data that cannot be extracted (step ST2; NO), the process from step ST1 until image data that can extract the feature point necessary for specifying the attributes and appearance of the vehicle is obtained. Repeat the process. In other words, the vehicle-mounted camera 10 captures the situation around the vehicle as a moving image, or shoots at a short shooting interval at which the same subject can be obtained, so that the subject can be specified accurately.

  When feature points necessary for specifying the attributes and appearance of the vehicle can be extracted (step ST2; YES), the vehicle specifying unit 12a extracts feature points from the image data of the vehicle that is the subject (step ST3). . As a result, feature information indicating the appearance of the vehicle as the subject (which may be a search key and may be a contour image of the vehicle obtained by binarization processing, for example) is generated.

Subsequently, the vehicle identification unit 12a compares the feature information generated in step ST3 with the pattern image corresponding to each attribute information in the built-in database 18, and determines whether there is attribute information that matches these. (Step ST4).
Here, when there is no attribute information that matches the two (step ST4; NO), the vehicle identification unit 12a instructs the auxiliary information acquisition unit 14 to refer to the external database, thereby attribute information corresponding to the feature information. Is acquired (step ST5).

When there is attribute information corresponding to the built-in database 18 (step ST4; YES), or when the attribute information is acquired from the external database in step ST5, the vehicle specifying unit 12a uses this attribute information as attribute information of the subject vehicle. As specified. When the attribute information is specified, the vehicle specifying unit 12a analyzes the image data of the vehicle that is the subject in more detail, and specifies appearance information at the time of shooting the vehicle. The process so far is the process of step ST6.
In addition, as vehicle attribute information, for example, vehicle type, car manufacturer name, brand name, vehicle type name, year of manufacture, body color name, grade name, displacement, drive system, car manufacturer country name, maximum horsepower, engine system Vehicle weight, maximum speed, fuel consumption, acceleration time of 0-100 km / h, and other information related to the specifications of the vehicle.
In addition, vehicle exterior information includes, for example, vehicle colors (including customized paints other than those in attributes), exterior (wheels, aero parts, stickers, seals, etc.), dirt, scratches, and outside the vehicle. There are interiors visible from the window (those visible from the window).

Next, in step ST7, the vehicle specifying unit 12a stores the specified attribute information and appearance information in the data storage area 13a of the storage unit 13 together with the shooting date / time information. In addition, when the position information of the own vehicle can be obtained, the information on the shooting location may be stored together.
Moreover, the vehicle specific | specification part 12a searches the log | history information of the same attribute with reference to the data storage area | region 13a. If there is history information with the same attribute, the vehicle identification unit 12a reads it from the data storage area 13a and transmits it to the content generation unit 15 together with the attribute information and appearance information acquired this time (step ST8).

The content generation unit 15 generates content including the attribute and appearance of the vehicle based on the vehicle attribute information and appearance information received from the vehicle specifying unit 12a.
For example, a My Album that registers vehicles identified from captured images one by one is created, and the degree of dirt, scratches, newness, and condition of the vehicle are registered in accordance with the attributes of each vehicle.
By doing in this way, the passenger can obtain in real time information on the vehicle passing the own vehicle while driving by referring to the album displayed on the information output unit 17 (in-vehicle display). In addition, a driver who has to focus on driving can know information about a vehicle that is different from the own vehicle after driving. In this case, the user can also grasp the location where the user encounters the vehicle by presenting the shooting location information.

In addition, when there is history information, the content generation unit 15 can count the appearance frequency of vehicles having the same attribute to obtain an encounter history with the vehicle.
For example, by creating and providing a quiz that combines past encounter frequencies, vehicle types, and other attribute information of vehicles of the same brand, passengers can effectively utilize the information and perform fun time killing.
Also, using the information registered in the above-mentioned my album, various vehicle information and the degree of care of the target vehicle are used as parameters, and points corresponding to these parameters are given. For example, passengers and other vehicles as card games It is also possible to realize a mode in which a battle game is played with other users who are on the road.

FIG. 4 is a flowchart showing a process for providing information about a person based on the vehicle periphery image data, and shows the process when the subject is a person.
When the subject specifying unit 12 inputs the image data of the subject extracted by the subject extracting unit 11, the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, and the sign specifying unit The image data is sorted and output to 12f and the feature specifying unit 12g. Here, the person specifying unit 12b determines that the type of the subject is a person (step ST1a).

Next, the person specifying unit 12b determines whether or not the image data of the subject is data that can extract feature points necessary for specifying the attributes and appearance of the person (step ST2a).
Here, if the feature points cannot be extracted (step ST2a; NO), the process from step ST1a is performed until image data that can extract the feature points necessary for specifying the attributes and appearance of the person is obtained. Repeat the process. In other words, the vehicle-mounted camera 10 captures the situation around the vehicle as a moving image, or shoots at a short shooting interval at which the same subject can be obtained, so that the subject can be specified accurately.

  When the feature points necessary for specifying the attributes and appearance of the person can be extracted (step ST2a; YES), the person specifying unit 12b extracts the feature points from the image data of the person who is the subject (step ST3a). . As a result, feature information indicating the appearance of the person who is the subject (which may be a search key and may be a contour image of a person obtained by binarization processing, for example) is generated.

Subsequently, the person specifying unit 12b compares the feature information generated in step ST3a with the pattern image corresponding to each attribute information in the built-in database 18, and determines whether there is attribute information that matches these. (Step ST4a). Here, when there is no attribute information that matches the two (step ST4a; NO), the person identifying unit 12b instructs the auxiliary information acquiring unit 14 to refer to the external database, and there is attribute information corresponding to the feature information. (Step ST5a). If there is no attribute information in the external database (step ST5a; NO), the user inputs attribute information about the person to the person specifying unit 12b using an input device (not shown) (step ST6a).
In the built-in database 18, detailed information regarding the person input by the user is registered as attribute information. For example, information related to the user's acquaintance (name, address, occupation, sex, age, race, etc.) is registered together with the image data of the person. If the subject person is a celebrity, detailed information can be acquired from an external database.

When attribute information matching the feature information generated in step ST3a is in the built-in database 18 (step ST4a; YES), or when attribute information matching the feature information generated in step ST3a is in the external database (step ST5a; YES) or when attribute information is input by the user in step ST6a, the person specifying unit 12b specifies this attribute information as attribute information of the person of the subject.
After specifying the attribute information, the person specifying unit 12b analyzes the image data of the person who is the subject in more detail and specifies the appearance information at the time of shooting the person. The process so far is the process of step ST7a.
The person attribute information includes, for example, name, sex, address, birthplace, date of birth, occupation, family structure, educational background, political thought, and other information related to the person.
The appearance information of a person includes, for example, a person's clothes, body shape, race, and the like.

Next, in step ST8a, the person specifying unit 12b stores the specified attribute information and appearance information in the data storage area 13a of the storage unit 13 together with the shooting date / time information. In addition, when the position information of the own vehicle can be obtained, the information on the shooting location may be stored together.
The person specifying unit 12b searches the history information with the same attribute with reference to the data storage area 13a. If there is history information with the same attribute, the person identification unit 12b reads it from the data storage area 13a and transmits it to the content generation unit 15 together with the attribute information and appearance information acquired this time (step ST9a).

The content generation unit 15 generates content including the attribute and appearance of the person based on the attribute information and appearance information of the person received from the person specifying unit 12b.
For example, content that provides detailed information of a celebrity or a person registered by a user is generated. In addition, by classifying or evaluating the fashion at the time of shooting, it is possible to directly grasp the fashion actually worn by pedestrians rather than indirectly from magazines, TV, the Internet, etc. Can do.

In addition, when there is history information, the content generation unit 15 can count the acquisition frequency of attribute information of the same person and use it as an encounter history with the person. You can use this encounter history to check the date and place you last met, or use it if you forget your name.
Furthermore, from the information on the clothes of the person photographed during the driving, for example, the transition of the long-sleeved wearing rate is used as a graph, and the content that visualizes the transition of the season can be compared with the usual year. In addition, by providing this content to users in other regions through communication, it is possible to provide reference information for clothing.

FIG. 5 is a flowchart showing a process of providing information about animals based on the vehicle periphery image data, and shows a process when the subject is an animal other than a human.
When the subject specifying unit 12 inputs the image data of the subject extracted by the subject extracting unit 11, the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, and the sign specifying unit The image data is sorted and output to 12f and the feature specifying unit 12g. Here, the animal specifying unit 12c determines that the type of subject is an animal (step ST1b).

Next, the animal specifying unit 12c determines whether or not the image data of the subject is data capable of extracting feature points necessary for specifying the attributes and appearance of the animal (step ST2b).
Here, when the feature point is image data that cannot be extracted (step ST2b; NO), until the image data that can extract the feature point necessary for specifying the attribute and appearance of the animal is obtained, the process from step ST1b is performed. Repeat the process. In other words, the vehicle-mounted camera 10 captures the situation around the vehicle as a moving image, or shoots at a short shooting interval at which the same subject can be obtained, so that the subject can be specified accurately.

  When the feature points necessary for specifying the attributes and appearance of the animal can be extracted (step ST2b; YES), the animal specifying unit 12c extracts the feature points from the image data of the animal that is the subject (step ST3b). . As a result, feature information indicating the appearance of the animal that is the subject (which may be a search key and may be an outline image of the animal obtained by binarization, for example) is generated.

Subsequently, the animal specifying unit 12c compares the feature information generated in step ST3b with the pattern image corresponding to each attribute information in the built-in database 18, and determines whether there is attribute information that matches the feature information. (Step ST4b). Here, when there is no attribute information that matches the two (step ST4b; NO), the animal identification unit 12c instructs the auxiliary information acquisition unit 14 to refer to the external database, and thereby the animal corresponding to the feature information is identified. It is determined whether there is a unique name or local attribute information for each individual (step ST5b). If this attribute information is not in the external database (step ST5b; NO), the user uses the input device (not shown) to send the attribute information (name (pet name), personality, etc.) regarding the animal to the animal identification unit 12c. Input (step ST6b).
In the built-in database 18, detailed information about animals is registered as attribute information. For example, information (name, address, sex, age, type, etc.) related to pets of users and neighbors is registered together with the image data of the animal.

When attribute information matching the feature information generated in step ST3b is in the built-in database 18 (step ST4b; YES), or attribute information matching the feature information generated in step ST3b is in the external database (step ST5b; YES) or when attribute information is input by the user in step ST6b, the animal specifying unit 12c specifies this attribute information as attribute information of the subject animal.
After specifying the attribute information, the animal specifying unit 12c analyzes the image data of the animal that is the subject in more detail and specifies the appearance information at the time of photographing the animal. The process so far is the process of step ST7b.
The animal attribute information includes, for example, a name (animal name), pet name, distribution area, owner name, address, and other information related to the animal.
The animal appearance information includes, for example, animal patterns, dirt conditions, and care conditions.

Next, in step ST8b, the animal specifying unit 12c stores the specified attribute information and appearance information in the data storage area 13a of the storage unit 13 together with the shooting date / time information. In addition, when the position information of the own vehicle can be obtained, the information on the shooting location may be stored together.
The animal specifying unit 12c searches the history information with the same attribute with reference to the data storage area 13a. If there is history information with the same attribute, the animal identification unit 12c reads it from the data storage area 13a and transmits it to the content generation unit 15 together with the attribute information and appearance information acquired this time (step ST9b).

The content generation unit 15 generates content including the attribute and appearance of the animal based on the attribute information and appearance information of the animal received from the animal identification unit 12c.
For example, the content of displaying a divided page by tagging a pet pattern, care condition, and pet clothes for each pet for each page is generated. By referring to this content, it is possible to know the trends of nearby pets.
In addition, when there is history information, the content generation unit 15 counts the acquisition frequency of attribute information of the same animal, and generates content that visualizes the distribution of the animal and content that visualizes the distribution of various animals. It is possible to know the distribution of specific animals and the ecosystem in the driven area.

FIG. 6 is a flowchart showing a process of providing information related to a flying object based on the vehicle periphery image data, and shows the process when the subject is a flying object.
When the subject specifying unit 12 inputs the image data of the subject extracted by the subject extracting unit 11, the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, and the sign specifying unit The image data is sorted and output to 12f and the feature specifying unit 12g. Here, the flying object specifying unit 12d determines that the type of the subject is a flying object (step ST1c).

Next, the flying object specifying unit 12d determines whether or not the image data of the subject is data capable of extracting feature points necessary for specifying the attributes and appearance of the flying object (step ST2c).
Here, if the feature points cannot be extracted (step ST2c; NO), from step ST1c until image data capable of extracting feature points necessary for specifying the attributes and appearance of the flying object is obtained. Repeat the process. In other words, the vehicle-mounted camera 10 captures the situation around the vehicle as a moving image, or shoots at a short shooting interval at which the same subject can be obtained, so that the subject can be specified accurately.

  When it is possible to extract the feature points necessary for specifying the attributes and appearance of the flying object (step ST2c; YES), the flying object specifying unit 12d extracts the feature points from the image data of the flying object that is the subject ( Step ST3c). As a result, feature information indicating the appearance of the flying object that is the subject (which may be a search key, for example, an outline image of the flying object obtained by binarization processing) is generated.

  Subsequently, the flying object specifying unit 12d compares the feature information generated in step ST3c with the pattern image corresponding to each attribute information in the built-in database 18, and determines whether there is attribute information that matches the feature information. (Step ST4c). Here, when there is no attribute information that matches the two (step ST4c; NO), the flying object specifying unit 12d instructs the auxiliary information acquisition unit 14 to refer to the external database, thereby flying corresponding to the feature information. The attribute information of an object is acquired (step ST5c).

When the attribute information matching the feature information generated in step ST3c is in the built-in database 18 (step ST4c; YES), or when the attribute information is acquired from the external database in step ST5c, the flying object specifying unit 12d The attribute information is specified as the attribute information of the flying object of the subject.
After specifying the attribute information, the flying object specifying unit 12d analyzes the image data of the flying object that is the subject in more detail, and specifies appearance information at the time of shooting the flying object. The process so far is the process of step ST6c.
As the attribute information of the flying object, for example, when the flying object is an aircraft, there are an aircraft type, a manufacturer name, a manufacturing year, an airline, a route name, and other information related to the aircraft. When the flying object is a bird, there are a bird name, ecological information, a distribution area, and the like.
Further, the appearance information of the flying object includes, for example, an outer surface painting of an aircraft, and in the case of a bird, there are a pattern, a dirt condition, a care condition, and the like.

Next, in step ST7c, the flying object specifying unit 12d stores the specified attribute information and appearance information together with the shooting date / time information in the data storage area 13a of the storage unit 13. In addition, when the position information of the own vehicle can be obtained, the information on the shooting location may be stored together.
In addition, the flying object specifying unit 12d searches the history information having the same attribute with reference to the data storage area 13a. If there is history information with the same attribute, the flying object specifying unit 12d reads it from the data storage area 13a and transmits it to the content generation unit 15 together with the attribute information and appearance information acquired this time (step ST8c).

The content generation unit 15 generates content including the attributes and appearance of the flying object based on the flying object attribute information and appearance information received from the flying object specifying unit 12d.
For example, a My Album that registers the aircraft identified from the captured image one by one is created, and the exterior of the aircraft is registered according to the attributes of each aircraft. By doing in this way, the passenger can refer to the album displayed on the information output unit 17 (in-vehicle display) in real time while driving, information on the aircraft that can be visually recognized from the own vehicle. In addition, a driver who has to concentrate on driving can know information about an aircraft that can be visually recognized from the own vehicle after driving.
Further, if the flying object does not match any image data associated with the attribute information, the auxiliary information acquisition unit 14 may be instructed to upload an image of the flying object as an unidentified flying object on the net. .
Furthermore, when there is history information, the content generation unit 15 counts the frequency of acquiring attribute information of the same bird, and generates content that visualizes the distribution of the birds and content that visualizes the distribution of various birds. By providing it, it is possible to know the distribution of specific birds and the wild bird ecosystem in the driven area.

FIG. 7 is a flowchart showing a process for providing information about a plant based on the vehicle peripheral image data, and shows a process when the subject is a plant.
When the subject specifying unit 12 inputs the image data of the subject extracted by the subject extracting unit 11, the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, and the sign specifying unit The image data is sorted and output to 12f and the feature specifying unit 12g. Here, the plant specifying unit 12e determines that the type of the subject is a plant (step ST1d).

Next, the plant specifying unit 12e determines whether or not the image data of the subject is data that can extract feature points necessary for specifying the attributes and appearance of the plant (step ST2d).
Here, when the feature point is image data that cannot be extracted (step ST2d; NO), until the image data from which the feature point necessary for specifying the attribute and appearance of the plant can be obtained is obtained from step ST1d. Repeat the process. In other words, the vehicle-mounted camera 10 captures the situation around the vehicle as a moving image, or shoots at a short shooting interval at which the same subject can be obtained, so that the subject can be specified accurately.

  When the feature points necessary for specifying the attributes and appearance of the plant can be extracted (step ST2d; YES), the plant specifying unit 12e extracts the feature points from the image data of the plant that is the subject (step ST3d). . As a result, feature information indicating the appearance of the subject plant (which may be a search key, for example, may be a contour image of the plant obtained by binarization processing) is generated.

  Subsequently, the plant specifying unit 12e compares the feature information generated in step ST3d with the pattern image corresponding to each attribute information in the built-in database 18, and determines whether there is attribute information that matches these. (Step ST4d). Here, when there is no attribute information that matches the two (step ST4d; NO), the plant identification unit 12e instructs the auxiliary information acquisition unit 14 to refer to the external database, thereby identifying the plant corresponding to the feature information. Attribute information is acquired (step ST5d).

When the attribute information that matches the feature information generated in step ST3d is in the internal database 18 (step ST4d; YES), or when the attribute information is acquired from the external database in step ST5d, the plant specifying unit 12e The information is specified as the attribute information of the subject plant.
After specifying the attribute information, the plant specifying unit 12e analyzes the image data of the plant that is the subject in more detail, and specifies the appearance information at the time of photographing the plant. The process so far is the process of step ST6d.
The plant attribute information includes, for example, plant type, name, distribution area, flowering time, and other information related to the plant.
The plant appearance information includes, for example, the color of flowers and leaves, the condition of flowering, and the condition of autumn leaves.

Next, in step ST7d, the plant specifying unit 12e stores the specified attribute information and appearance information in the data storage area 13a of the storage unit 13 together with the shooting date / time information. In addition, when the position information of the own vehicle can be obtained, the information on the shooting location may be stored together.
In addition, the plant specifying unit 12e searches for history information having the same attribute with reference to the data storage area 13a. If there is history information with the same attribute, the plant identification unit 12e reads it from the data storage area 13a and transmits it to the content generation unit 15 together with the attribute information and appearance information acquired this time (step ST8d).

The content generation unit 15 generates content including the attribute and appearance of the plant based on the plant attribute information and appearance information received from the plant identification unit 12e.
For example, when various kinds of plants (cherry blossoms, sunflowers, camellias) serving as seasonal information are detected, attribute information and contents showing the appearance are provided in correspondence with the seasons. As a result, cherry blossoms can be obtained in real time while driving. Further, the flowering information of cherry blossoms may be instructed to the auxiliary information acquisition unit 14 to upload the information on the net.
In addition, when there is history information, the content generation unit 15 counts the acquisition frequency of attribute information of the same plant, and generates content that visualizes the distribution of the plant and content that visualizes the distribution of various plants. It is possible to know the distribution of plants in the driven area.

FIG. 8 is a flowchart showing a process of providing information related to a sign based on the vehicle periphery image data, and shows a process when the subject is a sign.
When the subject specifying unit 12 inputs the image data of the subject extracted by the subject extracting unit 11, the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, and the sign specifying unit The image data is sorted and output to 12f and the feature specifying unit 12g. Here, the sign identifying unit 12f determines that the type of the subject is a sign (step ST1e).

Next, the sign specifying unit 12f determines whether or not the image data of the subject is data capable of extracting feature points necessary for specifying the attribute and appearance of the sign (step ST2e).
Here, if the feature point is image data that cannot be extracted (step ST2e; NO), until the image data from which the feature point necessary for specifying the attribute and appearance of the sign can be obtained is obtained from step ST1e. Repeat the process. In other words, the vehicle-mounted camera 10 captures the situation around the vehicle as a moving image, or shoots at a short shooting interval at which the same subject can be obtained, so that the subject can be specified accurately.

  When it is possible to extract the feature points necessary for specifying the attribute and appearance of the sign (step ST2e; YES), the sign specifying unit 12f extracts the feature points from the image data of the sign that is the subject (step ST3e). . As a result, feature information indicating the appearance of the sign as a subject (for example, an image showing the outline and design of the sign after binarization) is generated.

  Subsequently, the sign specifying unit 12f compares the feature information generated in step ST3e with the pattern image corresponding to each attribute information in the built-in database 18, and determines whether there is attribute information that matches the feature information. (Step ST4e). Here, when there is no attribute information that matches the two (step ST4e; NO), the sign specifying unit 12e instructs the auxiliary information acquisition unit 14 to refer to the external database, thereby indicating the sign corresponding to the feature information. Attribute information is acquired (step ST5e).

When attribute information that matches the feature information generated in step ST3e is in the internal database 18 (step ST4e; YES), or when attribute information is acquired from the external database in step ST5e, the sign specifying unit 12f The information is specified as attribute information of the sign of the subject.
After specifying the attribute information, the sign specifying unit 12f analyzes the image data of the sign that is the subject in more detail and specifies the appearance information at the time of photographing the sign. The process so far is the process of step ST6e.
The attribute information of the sign includes, for example, the kind of sign, name, meaning of the sign, and other information related to the sign.
Further, the appearance information of the sign includes, for example, the sign design and the degree of dirt.

Next, in step ST7e, the sign specifying unit 12f stores the specified attribute information and appearance information together with the shooting date / time information in the data storage area 13a of the storage unit 13. In addition, when the position information of the own vehicle can be obtained, the information on the shooting location may be stored together.
Also, the sign specifying unit 12f searches for history information having the same attribute with reference to the data storage area 13a. If there is history information with the same attribute, the sign specifying unit 12f reads it from the data storage area 13a and transmits it to the content generation unit 15 together with the attribute information and appearance information acquired this time (step ST8e).

The content generation unit 15 generates content including the attribute and appearance of the sign based on the attribute information and appearance information of the sign received from the sign specifying unit 12f.
For example, when there is history information, the content generation unit 15 generates content that provides information on variable median strips in a plurality of lanes that change according to time and time, together with shooting date and time. By viewing this content, the user can grasp the situation of the road while driving. It is also possible to make this content public on the Web as reference information for a traveling vehicle that is about to travel on the road.

FIG. 9 is a flowchart showing a process for providing information about a feature based on vehicle peripheral image data, and shows a process when the subject is a feature.
When the subject specifying unit 12 inputs the image data of the subject extracted by the subject extracting unit 11, the vehicle specifying unit 12a, the person specifying unit 12b, the animal specifying unit 12c, the flying object specifying unit 12d, the plant specifying unit 12e, and the sign specifying unit The image data is sorted and output to 12f and the feature specifying unit 12g. Here, the feature identifying unit 12g determines that the type of the subject is a feature (step ST1f).

Next, the feature specifying unit 12g determines whether or not the image data of the subject is data capable of extracting feature points necessary for specifying the attributes and appearance of the features (step ST2f).
Here, if the feature point is image data that cannot be extracted (step ST2f; NO), from step ST1f until image data that can extract the feature point necessary for specifying the attribute and appearance of the feature is obtained. Repeat the process. In other words, the vehicle-mounted camera 10 captures the situation around the vehicle as a moving image, or shoots at a short shooting interval at which the same subject can be obtained, so that the subject can be specified accurately.

  When the feature points necessary for specifying the attributes and appearance of the feature can be extracted (step ST2f; YES), the feature specifying unit 12g extracts the feature points from the image data of the feature that is the subject ( Step ST3f). Thereby, feature information indicating the appearance of the feature that is the subject (for example, the contour image of the feature by binarization processing) is generated.

  Subsequently, the feature specifying unit 12g compares the feature information generated in step ST3f with the pattern image corresponding to each attribute information in the built-in database 18, and determines whether there is attribute information that matches the feature information. (Step ST4f). Here, when there is no attribute information that matches the two (step ST4f; NO), the feature specifying unit 12g instructs the auxiliary information acquisition unit 14 to refer to the external database, thereby corresponding to the feature information. The attribute information of an object is acquired (step ST5f).

When the attribute information that matches the feature information generated in step ST3f is in the built-in database 18 (step ST4f; YES), or when the attribute information is acquired from the external database in step ST5f, the feature specifying unit 12g The attribute information is specified as the attribute information of the feature of the subject.
After specifying the attribute information, the feature specifying unit 12g analyzes the image data of the feature that is the subject in more detail and specifies the appearance information at the time of photographing the feature. The process so far is the process of step ST6f.
The feature attribute information includes, for example, a building type, name, construction date, and other information related to the feature when the feature is a building. In addition, when the feature is a terrain such as a mountain or a river, its name, height, depth, length, and the like are included.
Further, as the appearance information of the feature, for example, when the feature is a building, there are the appearance of the building, the degree of dirt, the degree of care, and the like. In addition, when the feature is a terrain such as a mountain or a river, there is appearance information associated with the weather such as its shape, the remaining snow condition, and the state of being hidden in the clouds.

Next, in step ST7f, the feature specifying unit 12g stores the specified attribute information and appearance information together with the shooting date / time information in the data storage area 13a of the storage unit 13. In addition, when the position information of the own vehicle can be obtained, the information on the shooting location may be stored together.
The feature specifying unit 12g searches the history information with the same attribute with reference to the data storage area 13a. If there is history information with the same attribute, the feature specifying unit 12g reads it from the data storage area 13a and transmits it to the content generation unit 15 together with the attribute information and appearance information acquired this time (step ST8f).

The content generation unit 15 generates content including the attribute and appearance of the feature based on the attribute information and appearance information of the feature received from the feature specifying unit 12g.
For example, a my album for registering buildings identified from the photographed image one by one is created and registered together with attributes for each type of building. By doing in this way, the passenger can refer to the album displayed on the information output unit 17 (in-vehicle display) in real time while driving, with respect to the building that is visible from the own vehicle. In addition, by generating content that provides a quiz such as “how many temples have passed since the start of driving”, it is possible to learn the state of the driving region.

In the above description, the case where the content generation unit 15 creates content by a preset information providing method has been described. However, the content may be generated by executing the application provided from the application provision unit 16. .
For example, when the content generation unit 15 generates content based on the feature attribute information and appearance information received from the feature specifying unit 12g, the application providing unit 16 realizes a desired information providing method. Receive the offer.
As such an application, an application that provides entertainment content for estimating attribute information from the texture or shape of a building can be considered.
For example, there is interactive content that presents the question “What is the architectural style of the church that passed by?” And makes the user answer.
Alternatively, the application may be an application that creates a user's My Collection by overlooking the shape of the mountain and recording a captured image and date and time when a mountain close to Mt. Fuji is extracted.
Note that the application requested by the content generation unit 15 to the application providing unit 16 may be set in the content generation unit 15 by the user using an input device (not shown).

  As described above, according to the first embodiment, the subject extraction unit 11 that inputs the image data around the vehicle photographed by the in-vehicle camera 10 and extracts the image of the main subject from the image data, and the subject type The type of the subject is determined from the built-in database 18 that stores attribute information about the subject and the subject image extracted by the subject extraction unit 11, and the attribute information stored in the built-in database 18 is determined based on the determined type. The subject specifying unit 12 for specifying the subject's attribute information and specifying the appearance information indicating the appearance of the subject at the time of photographing from the subject image, and the subject's attribute information and appearance information specified by the subject specifying unit 12 Based on the content generation unit 15 that generates content including the attributes and appearance of the subject and provides them to the user in the vehicle, Content Ceiling generating unit 15 has generated and an information output section 17 to be provided to the user in the vehicle. By configuring in this way, it is possible to provide information on the situation around the vehicle that changes from moment to moment to the user in the vehicle.

  In addition, according to the first embodiment, the application generation unit 16 includes an application that realizes various information providing methods and provides an application in response to a request from the content generation unit 15. Then, the application acquired from the application providing unit 16 is executed to generate content of the information providing method corresponding to the application. By comprising in this way, the information regarding the condition around a vehicle can be provided to the user in a vehicle in various aspects.

  Further, according to the first embodiment, the content generation unit includes the storage unit 13 that associates the attribute information, the appearance information, and the shooting date / time information of the subject specified by the subject specifying unit 12 and stores them as history information of the subject. 15, based on the history information of the subject stored in the storage unit 13, generates content including the attribute and appearance of the subject and the transition with the passage of time. By comprising in this way, the transition with the time passage of the information regarding the condition around a vehicle can be provided to the user in a vehicle.

  Furthermore, according to the first embodiment, the auxiliary information acquisition unit 14 that acquires attribute information by communicating with an external database that stores attribute information about the subject for each type of subject is provided, and the subject specifying unit 12 includes: If the built-in database 18 does not have attribute information corresponding to the determined type, the auxiliary information acquisition unit 14 is instructed to acquire attribute information corresponding to the type from the external database. With this configuration, it is possible to accurately acquire attribute information corresponding to the type of subject determined from the image.

  Furthermore, according to the first embodiment, the subject specifying unit 12 determines at least one of a vehicle, a person, an animal, a flying object, a plant, a sign, and a feature as a subject type. Information about the object can be provided to the user in the vehicle.

Embodiment 2. FIG.
FIG. 10 is a diagram showing an overview of the in-vehicle information providing system according to Embodiment 2 of the present invention. In the vehicle information provision system shown in FIG. 10, the in-vehicle device 100 performs information provision processing in cooperation with at least one of the portable information terminal 101 such as a smartphone and the server device 102. Hereinafter, a configuration aspect of the in-vehicle information providing system will be described.

First, a case where the in-vehicle device 100 functions as an in-vehicle information providing device in cooperation with the server device 102 will be described.
In this configuration, it is conceivable that the in-vehicle device 100 communicates directly with the server device 102, or the in-vehicle device 100 communicates with the server device 102 via the portable information terminal 101. The server apparatus 102 includes the subject extracting unit 11, the subject specifying unit 12, the storage unit 13, and the application providing unit 16 described in the first embodiment. The in-vehicle device 100 also provides the content generation unit 15 that generates content using the subject attribute information and appearance information specified by the subject specifying unit 12 of the server device 102, and the content generated by the content generation unit 15 to the user. The information output part 17 for performing at least is provided.

Further, the in-vehicle device 100 basically has only a communication function, and may receive a content generation result by the server device 102 and provide it to the user.
Also in this configuration, the in-vehicle device 100 communicates directly with the server device 102, or the in-vehicle device 100 communicates with the server device 102 via the portable information terminal 101.
However, unlike the above configuration, the in-vehicle device 100 includes the information output unit 17 but does not include the content generation unit 15.
That is, the server apparatus 102 includes a subject extraction unit 11, a subject specification unit 12, a storage unit 13, a content generation unit 15, and an application provision unit 16.

In addition, a case where the portable information terminal 101 performs content generation processing in cooperation with the server device 102 and the in-vehicle device 100 provides the result to the user will be described.
For example, the portable information terminal 101 includes the subject extraction unit 11, the subject identification unit 12, the content generation unit 15, and the application provision unit 16 described in the first embodiment, and the server device 102 includes the built-in database 18 and the storage unit 13. Is provided. The in-vehicle device 100 includes an information output unit 17 in addition to the communication function.
Even if configured as described above, the same effect as in the first embodiment can be obtained.

  In the present invention, within the scope of the invention, any combination of each embodiment, any component of each embodiment can be modified, or any component can be omitted in each embodiment. .

  DESCRIPTION OF SYMBOLS 1 In-vehicle information provision apparatus, 10 In-vehicle camera, 11 Subject extracting part, 12 Subject specifying part, 12a Vehicle specifying part, 12b Person specifying part, 12c Animal specifying part, 12d Flying object specifying part, 12e Plant specifying part, 12f Marking specifying part , 12g feature identification unit, 13 storage unit, 13a data storage area, 14 auxiliary information acquisition unit, 15 content generation unit, 16 application provision unit, 17 information output unit, 18 built-in database.

Claims (5)

A subject extraction unit that inputs image data around the vehicle captured by the in-vehicle camera and extracts an image of a main subject from the image data;
A database for storing attribute information about the subject for each subject type;
The type of the subject is determined from the image of the subject extracted by the subject extraction unit, the attribute information of the subject is specified from the attribute information stored in the database based on the determined type, and the subject A subject identifying unit that identifies appearance information indicating an appearance of the subject at the time of shooting from the image;
A content generating unit that generates content including the attribute and appearance of the subject based on the attribute information and appearance information of the subject specified by the subject specifying unit;
An in-vehicle information providing apparatus comprising: an information output unit that provides content generated by the content generation unit to a user in the vehicle. It has an application that realizes various information providing methods, and includes an application providing unit that provides the application in response to a request from the content generation unit,
The in-vehicle information providing apparatus according to claim 1, wherein the content generation unit executes an application acquired from the application providing unit and generates content of an information providing method corresponding to the application. A storage unit that associates attribute information, appearance information, and shooting date / time information of the subject specified by the subject specifying unit and stores the information as history information of the subject;
The content generation unit generates content including an attribute and an appearance of the subject and a transition with the passage of time based on history information of the subject stored in the storage unit. The in-vehicle information providing apparatus according to claim 1 or 2. An information acquisition unit that acquires the attribute information by communicating with an external database that stores attribute information about the subject for each type of subject,
The subject specifying unit, when there is no attribute information corresponding to the determined type in the database, instructs the information acquisition unit to acquire attribute information corresponding to the type from the external database. The in-vehicle information providing apparatus according to any one of claims 1 to 3.   5. The device according to claim 1, wherein the subject specifying unit determines at least one of a vehicle, a person, an animal, an aircraft, a plant, a sign, and a feature as a subject type. In-vehicle information providing device.

Images