WO2020202346A1

WO2020202346A1 - Information provision system and information terminal

Info

Publication number: WO2020202346A1
Application number: PCT/JP2019/014253
Authority: WO
Inventors: 山岡　大祐; 田中　一彦; 祐瀧口; 瞳 ▲濱▼村
Original assignee: 本田技研工業株式会社
Priority date: 2019-03-29
Filing date: 2019-03-29
Publication date: 2020-10-08
Also published as: CN113614709A; JPWO2020202346A1

Abstract

An information provision system which uses an information terminal having a camera and a display to provide information acquired from a server to a user, wherein the server stores, in association with each article of a plurality of types of articles, posted information that relates to use of the article and that is sent from users, and wherein the information terminal comprises: a display control means which displays, on the display, a captured image acquired by the camera; an identification means which identifies an article included in the captured image displayed on the display; and an acquisition means which acquires, from the server, posted information that is associated with the article identified by the identification means, and that is stored in the server. The display control means displays, on the display, an augmented reality image of the posted information acquired by the acquisition means, in such a manner that the augmented reality image is superimposed on the captured image acquired by the camera.

Description

Information provision system and information terminal

The present invention relates to an information providing system that provides information to a user, an information terminal, and an information terminal that posts information to a server.

An application that provides services by connecting to a network via a mobile terminal such as a smartphone has been devised. For example, in Patent Document 1, a captured image is displayed on a display in a portable wireless communication terminal (smartphone), and guidance (name) of a component included in the captured image is superimposed and displayed on the display. It is disclosed that the operation manual of the component is displayed on the display when the guidance of the superimposed component is pressed.

Japanese Unexamined Patent Publication No. 2014-215845

The user may desire information that is not described in the operation manual, such as a method for using the in-vehicle part more comfortably, a design change of the part, or addition of a function. However, it is difficult to obtain such information appropriately and easily without knowing the model of the part even if the search site is used.

Therefore, an object of the present invention is to allow the user to easily acquire information for expanding the range of use of the article.

The information providing system as one aspect of the present invention is an information providing system that provides a user with information acquired from a server by using an information terminal having a camera and a display, and the server is a plurality of types of articles. The information terminal stores the posted information sent from the user regarding the use of each article in association with each article, and the information terminal includes a display control means for displaying a photographed image obtained by the camera on the display and the display. Acquisition of the specific means for specifying the article included in the photographed image displayed in the above and the post information stored in the server associated with the article specified by the specific means from the server. The display control means has means, and the display control means superimposes an expanded reality image of the posted information acquired by the acquisition means on the captured image obtained by the camera and displays it on the display. It is a feature.

According to the present invention, for example, regarding the use of an article, information not described in the operation manual can be provided to the user through augmented reality, so that the user can expand the range of use of the article.

The accompanying drawings are included in the specification, which form a part thereof, show an embodiment of the present invention, and are used together with the description to explain the principle of the present invention.
Block diagram showing the configuration of the information provision system Diagram showing an example of community ideas Diagram showing an example of a data tree of posted information stored in a server device Flowchart showing processing when the information providing program is executed A diagram showing an image displayed on the display of an information terminal when the information providing program is executed. A diagram showing an image displayed on the display of an information terminal when the information providing program is executed. Diagram showing an example of applying an information provision program to a motorcycle Diagram showing an example of applying an information provision program to a cultivator Flowchart showing the process when the posting program is executed Diagram showing the image displayed on the display of the information terminal when the posting program is executed. Diagram showing the image displayed on the display of the information terminal when the posting program is executed. Flowchart showing the registration process of useful information

Hereinafter, embodiments will be described in detail with reference to the attached drawings. It should be noted that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more of the plurality of features described in the embodiments may be arbitrarily combined. In addition, the same or similar configuration will be given the same reference number, and duplicate description will be omitted.

<First Embodiment>
The first embodiment according to the present invention will be described. FIG. 1 is a block diagram showing a configuration of the information providing system 100 of the present embodiment. The information providing system 100 includes, for example, an information terminal 10 and a server device 20 that are communicably connected to each other via a network NTW, and the information terminal 10 is used to acquire posted information stored in the server device 20 to be a user. It is a system to provide to. Here, the posted information is information sent from a user (third party, poster) who is using the article regarding the use of the article. The article includes, for example, the vehicle itself as a whole, parts mounted on the vehicle (vehicle-mounted parts), and the like, and in the present embodiment, the vehicle-mounted parts will be illustrated as articles. In the case of the present embodiment, the posted information includes a method of using the in-vehicle parts more comfortably, a method of changing the design (for example, color, pattern, pattern) of the in-vehicle parts, a method of adding the functions of the in-vehicle parts, and the like. This is information posted by the user of the in-vehicle component about "a mode in which the in-vehicle component is used" that is not described in the operation manual of the in-vehicle component. Hereinafter, the "utilization mode" may be referred to as a "usage mode". Further, in the present embodiment, as an in-vehicle component, a component mounted on a four-wheeled vehicle will be described as an example.

First, the configuration of the information terminal 10 will be described. The information terminal 10 may include, for example, a processing unit 11, a storage unit 12, a camera 13, a display 14, a position detection sensor 15, a posture detection sensor 16, and a communication unit 17. Each part of the information terminal 10 is connected to each other so as to be able to communicate with each other via the system bus 18. Examples of the information terminal 10 include a smartphone and a tablet terminal. In the present embodiment, an example in which a smartphone is used as the information terminal 10 will be described. Smartphones and tablet terminals are mobile terminals having various functions other than the call function, but the dimensions of the displays are different from each other. In general, tablet terminals have larger display dimensions than smartphones.

The processing unit 11 includes a processor typified by a CPU, a storage device such as a semiconductor memory, an interface with an external device, and the like. The storage unit 12 stores a program executed by the processor, data used by the processor for processing, and the like, and the processing unit 11 reads the program stored in the storage unit 12 into a storage device such as a memory and executes the program. be able to. In the case of the present embodiment, the storage unit 12 stores an application program (information providing program) for acquiring the posted information stored in the server device 20 and providing it to the user, and the processing unit 11 stores it. The information providing program stored in the unit 12 can be read into a storage device such as a memory and executed.

The camera 13 has a lens and an image sensor, and captures a subject to acquire a captured image. The camera 13 may be provided, for example, on an outer surface opposite to the outer surface on which the display 14 is provided. In addition, the display 14 notifies the user of information by displaying an image. In the case of the present embodiment, the display 14 can display the captured image acquired by the camera 13 in real time. Here, the display 14 of the present embodiment includes, for example, a touch panel type LCD (Liquid Crystal Display), and has a function of receiving information input from a user in addition to a function of displaying an image. However, the present invention is not limited to this, and the display 14 may have only the function of displaying an image, and an input unit (for example, a keyboard, a mouse, etc.) may be provided independently of the display 14.

The position detection sensor 15 detects the position and orientation of the information terminal 10. As the position detection sensor 15, for example, a GPS sensor that receives a signal from a GPS satellite and acquires the position information of the information terminal 10 at the present time, or a camera 13 of the information terminal 10 based on geomagnetism or the like is directed. An orientation sensor or the like that detects the orientation can be used. In the present embodiment, when the description "position of the information terminal 10" is described, it is assumed that the orientation of the information terminal 10 is included in addition to the position of the information terminal 10. In addition, the posture detection sensor 16 detects the posture of the information terminal 10. As the posture detection sensor 16, for example, an acceleration sensor, a gyro sensor, or the like can be used.

The communication unit 17 is communicably connected to the server device 20 via the network NTW. Specifically, the communication unit 17 has a function as a transmission unit that transmits information to the server device 20 via the network NTW and a function as a reception unit that receives information from the server device 20 via the network NTW. Have. In the case of the present embodiment, the communication unit 17 transmits information (component information) indicating the type and model of the vehicle-mounted component specified in the captured image obtained by the camera 13 to the server device 20. In addition, the communication unit 17 receives the posted information stored in the server device 20 in association with the specified in-vehicle component from the server device 20.

As a specific configuration of the processing unit 11, for example, a first acquisition unit 11a, a specific unit 11b, a second acquisition unit 11c, and a display control unit 11d may be provided. The first acquisition unit 11a acquires the data of the captured image acquired by the camera 13. The identification unit 11b analyzes the captured image by performing image processing such as a pattern matching method, and identifies an in-vehicle component included in the captured image displayed on the display 14. The second acquisition unit 11c acquires the posted information stored in the server device 20 in association with the in-vehicle component specified by the specific unit 11b from the server device 20 via the communication unit 17. The display control unit 11d displays the captured image acquired by the first acquisition unit 11a on the display 14. Further, when the posted information is acquired by the second acquisition unit 11, the display control unit 11e is concerned based on the position and attitude of the information terminal 10 detected by the position detection sensor 15 and the attitude detection sensor 16, respectively. The posted information is superimposed on the captured image and displayed on the display 14 as an Augmented Reality (AR) image.

Next, the configuration of the server device 20 will be described. The server device 20 may include a processing unit 21, a storage unit 22, and a communication unit 23. The processing unit 21 includes a processor typified by a CPU, a storage device such as a semiconductor memory, an interface with an external device, and the like. The storage unit 22 stores a program executed by the processor, data used by the processor for processing, and the like, and the processing unit 21 reads the program stored in the storage unit 22 into a storage device such as a memory and executes the program. can do. Further, the communication unit 23 is communicably connected to the information terminal 10 via the network NTW. Specifically, the communication unit 23 has a function as a receiving unit that receives information from the information terminal 10 via the network NTW and a function as a transmitting unit that transmits information to the information terminal 10 via the network NTW. Have.

In the case of the present embodiment, the server device 20 is a user (third party) different from the manufacturer of a plurality of types (for example, various types of vehicles) or a plurality of types (for example, various types of) in-vehicle parts. Person, contributor) posts information on how to use vehicles and in-vehicle parts. Such information is, for example, information that is not described in the operation manual of the in-vehicle parts and is useful for comfortably using the in-vehicle parts, and is referred to as "useful information" in the present embodiment. There is. For example, "useful information" can be posted to the server device 20 for each community composed of users of various vehicles. FIG. 2 is a diagram showing the concept of a community. As shown in FIG. 2, when the community 30a of the user 31 of the vehicle A, the community 30b of the user 31 of the vehicle B, and the community 30c of the user 31 of the vehicle C exist, each belonging to each of the communities 30a to 30c. The user 31 can post "useful information" in various vehicles to the server device 20.

The "useful information" posted by the user in this way is stored as posted information in the storage unit 22 of the server device 20 in association with each vehicle and each in-vehicle component. FIG. 3 is a diagram showing an example of a data tree of posted information stored in the storage unit 22 of the server device 20. In the example shown in FIG. 3, the type of vehicle (vehicles A, B, C ...) Is a major item, and the type of in-vehicle parts (in-vehicle parts a, b, c ...) Is a medium item. Multiple posted information is associated with in-vehicle parts as small items. For example, for the vehicle A, when the in-vehicle part a is used as the armrest portion of the driver's seat, "the usage mode of the armrest portion when sitting on the seat in a comfortable posture" and "the usage mode of the armrest portion when hooking a garbage bag". The posted information such as is associated with the armrest portion and stored in the storage unit 22. In addition, when the in-vehicle part b is used as the headrest of the driver's seat, "design change and addition of functions" such as a method of changing the color of the headrest, a method of changing to a headrest with a pattern or a picture, and a method of painting a character on the headrest. Post information such as "Usage mode of the headrest when performing the above" is stored in the storage unit 22 in association with the headrest. Similarly, when the vehicle-mounted component c is a rear side window, posted information such as "usage mode of the rear side window when attaching a curtain" is stored in the storage unit 22 in association with the rear side window. Here, the posted information may include a photograph, a video, an illustration, a voice, and the like in addition to the text document.

[Processing at information terminal]
Next, the processing performed by the information terminal 10 when the information providing program is executed will be described with reference to FIGS. 4 and 5A to 5B. FIG. 4 is a flowchart showing processing performed by the processing unit 11 of the information terminal 10. 5A to 5B are diagrams showing images displayed on the display 14 of the information terminal 10, and can be used to explain the processing performed by the processing unit 11 of the information terminal 10. In the following, an example will be described in which the inside of the vehicle is photographed by the camera 13 of the information terminal 10 and the posted information of the in-vehicle parts provided in the vehicle is displayed on the display as an augmented reality image.

In S11, the processing unit 11 (first acquisition unit 11a) causes the camera 13 to start shooting and acquires a shot image from the camera 13. In S12, the processing unit 11 (display control unit 11d) sequentially displays the captured images acquired from the camera 13 on the display 14. In the case of the present embodiment, the user photographs the inside of the vehicle from the passenger side by the camera 13 of the information terminal 10, and the photographed image obtained by the camera 13 is displayed on the display of the information terminal 10 as shown in FIG. 5A. Is displayed. The captured image shown in FIG. 5A includes, for example, a driver's seat 41 (front seat), a front side door 42, a rear side door 43, and the like.

In S13, the processing unit 11 (specific unit 11b) identifies the in-vehicle parts included in the captured image displayed on the display 14. For example, the processing unit 11 first recognizes the in-vehicle parts included in the captured image by performing known image processing. As an example of known image processing, a portion (feature point) having a feature amount such as a corner, a curvature, a change in brightness, or a change in color is detected in a captured image, and the feature of the detected feature point is detected. An example is a method of recognizing in-vehicle parts from feature information indicating the quantity and positional relationship. The storage unit 12 of the information terminal 10 stores feature information for each of a plurality of types of vehicle-mounted components, and the processing unit 11 has a high degree of matching (that is, matching) of the feature information with respect to the recognized vehicle-mounted component. Investigate (determine) whether or not there are in-vehicle parts (the degree exceeds a predetermined value). As a result, the processing unit 11 can specify the model of the vehicle-mounted component included in the captured image, the model of the vehicle on which the vehicle-mounted component is mounted, and the like.

In S14, the processing unit 11 determines whether or not the vehicle-mounted component is specified in the captured image displayed on the display 14. If the vehicle-mounted component is specified in the captured image, the process proceeds to S15, and if the vehicle-mounted component is not specified in the captured image, the process returns to S13. In the following, a case where the armrest portion 41a and the headrest 41b provided in the driver's seat and the seat belt 44 for the driver's seat are specified as in-vehicle parts will be described as an example.

In S15, the processing unit 11 (second acquisition unit 11c) acquires the posted information stored in the storage unit 22 of the server device 20 in association with the in-vehicle component specified in S13 from the server device 20. For example, the processing unit 11 transmits component information indicating the type and model of the vehicle-mounted component specified in S13 to the server device 20 via the communication unit 17. The server device 20 that has received the component information searches for the posted information that is associated with the in-vehicle component and stored in the storage unit 22 based on the component information, and transmits the searched posted information to the information terminal 10. To do. As a result, the processing unit 11 can acquire the posted information regarding the in-vehicle component specified in S13.

In S16, the processing unit 11 (display control unit 11d) superimposes (superimposes) the posted information acquired in S15 on the captured image obtained by the camera 13 as an augmented reality image 50 and displays it on the display 14. .. At this time, the processing unit 11 moves the augmented reality image 50 of the posted information according to the movement of the information terminal 10 based on the position and orientation information of the information terminal 10 detected by the position detection sensor 15 and the attitude detection sensor 16. As described above, the image captured by the camera 13 and the augmented reality image 50 of the posted information are aligned. That is, the processing unit 11 uses the posted information as the augmented reality image 50 as the augmented reality image 50 so as to match the position of the in-vehicle component in the captured image displayed on the display 14 based on the position / orientation information of the information terminal 10. To display.

For example, when the posted information regarding the armrest portion 41a of the driver's seat is acquired from the server device 20 in S15, as shown in FIG. 5B, the processing unit 11 points to the augmented reality image 50a of the posted information and points to the armrest portion 41a. It is generated as a "speech balloon" and displayed on the display 14. At this time, in the processing unit 11, even when the position and orientation of the information terminal 10 are changed, the sensors 15 to 16 so that the "blowing" of the posted information as the augmented reality image 50a always points to the armrest 41a. Based on the information on the position and orientation of the information terminal 10 detected in the above, the augmented reality image 50a and the captured image are aligned. Similarly, when the posted information regarding the headrest 41b of the driver's seat is acquired from the server device 20, the processing unit 11 points to the augmented reality image 50b of the posted information as shown in FIG. 5B. It is generated as a "speech balloon" and displayed on the display 14. Here, the driver's seatbelt 44 was identified as an in-vehicle component in S15, but since the posted information is not stored in the server device 20, the posted information about the seatbelt is displayed on the display 14 in FIG. 5B. It has not been. In addition, it may be displayed that there is no posted information about the seat belt.

In S17, the processing unit 11 determines whether or not to terminate the information providing program. For example, when the processing unit 11 detects an instruction from the user to end the information providing program, or when the user turns off the power of the information terminal 10, the user turns off the display 14 of the information terminal 10. In some cases, it can be determined that the information provision program will be terminated. If it is determined that the information providing program is not terminated, the process returns to S13, and if it is determined that the information providing program is terminated, the program is terminated.

As described above, in the information providing system 100 of the present embodiment, the posted information posted by a third party for each of the plurality of types of in-vehicle parts is associated with each in-vehicle component and stored in the server device 20. Then, the information terminal 10 identifies the in-vehicle parts included in the captured image obtained by the camera 13, and also transmits the posted information associated with the specified in-vehicle parts and stored in the server device 20 from the server device 20. It is acquired and the posted information is displayed as an augmented reality image by superimposing it on the captured image. As a result, for example, when the user wants to obtain posted information (useful information) about a usage mode that is not described in the operation manual or the like for a predetermined in-vehicle part, the user takes a picture of the in-vehicle part with the camera 13 of the information terminal 10. Then, the posted information can be easily acquired (understood) through the augmented reality image. That is, the user can expand the range of use of the in-vehicle parts through augmented reality by utilizing the information providing system of the present embodiment.

<Second Embodiment>
A second embodiment according to the present invention will be described. In the first embodiment described above, an example in which an information providing system that provides a user with posted information stored in the server device 20 is applied to a part (vehicle-mounted part) mounted on a four-wheeled vehicle has been described, but the present invention is limited to this. It is not something that can be done. For example, the above-mentioned parts mounted on saddle-mounted vehicles (motorcycles, tricycles), or parts mounted on other machines such as ships, aircraft, cultivators, lawnmowers, generators, and snowplows. An information providing system can be provided. In the present embodiment, an example in which the above information providing system is applied to parts mounted on a motorcycle and a cultivator will be described.

First, an example of applying an information provision program to a motorcycle will be described. FIG. 6 is a diagram showing an example in which a photographed image obtained by photographing a motorcycle 45 by the camera 13 of the information terminal 10 is displayed on the display 14 of the information terminal 10. In the example shown in FIG. 6, in the captured image of the motorcycle 45 displayed on the display 14, the key insertion unit 45a and the sheet 45b are specified by the processing unit 11 (specification unit 11b), and the posted information thereof is an augmented reality image. As 50, it is displayed on the display 14 overlaid on the captured image. Examples of the posted information may be information on equipment such as a cover of the key insertion portion 45a, information on know-how such as a riding method with less fatigue, and the like. In addition, the user enriches the posted information about the same type of motorcycle by posting the information obtained by using the motorcycle 45 (article) to the server device 20, such as the knowledge and information noticed by using the motorcycle 45. , The community of users of the motorcycle can be expanded.

Next, an example of applying the information provision program to the cultivator will be described. FIG. 7 is a diagram showing an example in which a photographed image obtained by photographing the cultivator 46 by the camera 13 of the information terminal 10 is displayed on the display 14 of the information terminal 10. In the example shown in FIG. 7, the cultivator 46 itself (that is, the model of the cultivator) is specified by the processing unit 11 (specific unit 11b) in the captured image displayed on the display 14, and the posted information about the cultivator 46 is expanded. As the real image 50, it is displayed on the display 14 overlaid on the captured image. Examples of posted information may be, for example, information on know-how such as a method of efficiently cultivating by a cultivator, or information on crops grown by a user of a cultivator. Information about crops includes information on crop management tips and the timing and extent of events such as planting, fertilizing, disinfecting, and incorporating crops. In addition, the user can experience and deliverables obtained by using the cultivator 46 (article), such as knowledge and information noticed by using the cultivator 46, or information about crops grown by using the cultivator 46. By posting the information to the server device 20, the posted information about the same type of cultivator can be enriched and the community of users of the cultivator can be expanded.

<Third Embodiment>
A third embodiment according to the present invention will be described. In the third embodiment, the process of posting new "useful information" to the server device 20 by the poster (user) and the new "useful information" posted by the poster in the server device 20. The process of registering (memorizing) as posted information will be described. In the following, the information before being posted to the server device 20 by a third party is referred to as "useful information", and the information after being posted and stored in the server device 20 is referred to as "posted information". To do.

[Posting process of useful information]
First, a process of posting new "useful information" to the server device 20 will be described. Posting of "useful information" to the server device 20 can be performed using, for example, an information terminal such as a smartphone or a tablet terminal. Since the information terminal for posting "useful information" has the same configuration as the information terminal 10 described in the first embodiment, the following is an example of posting "useful information" using the information terminal 10. Will be described.

In the case of the present embodiment, the storage unit 12 of the information terminal 10 stores an application program (information posting program) for posting "useful information" to the server device 20, and the processing unit 11 stores the storage unit 11. The information posting program stored in 12 can be read into a storage device such as a memory and executed. When the information posting program is executed in this way, the information providing system 100 shown in FIG. 1 can function as an information posting system for posting information from the information terminal 10 to the server 20.

FIG. 8 is a flowchart showing a process performed on the information terminal 10 when the information posting program is executed (that is, a process for posting new "useful information" from the information terminal 10 to the server device 20). 9A to 9B are diagrams showing images displayed on the display 14 of the information terminal 10, and can be used to explain the processing performed by the processing unit 11 of the information terminal 10.

In S21, the processing unit 11 (first acquisition unit 11a) causes the camera 13 to start shooting and acquires a shot image from the camera 13. In S22, the processing unit 11 (display control unit 11d) sequentially displays the captured images acquired from the camera 13 on the display 14. In the case of the present embodiment, the user photographs the inside of the vehicle from the passenger side by the camera 13 of the information terminal 10, and the display of the information terminal 10 is obtained by the camera 13 as shown in FIG. 5A described above. The captured image is displayed.

In S23, the processing unit 11 (specific unit 11b) identifies the in-vehicle parts included in the captured image displayed on the display 14. Since the identification of the in-vehicle parts in this step is the same as the step of S13 described in the first embodiment, detailed description of this step will be omitted. In S24, the processing unit 11 superimposes the augmented reality image 51 of the selection button for the user to select the in-vehicle component specified in S23 on the captured image and displays it on the display 14. At this time, the processing unit 11 matches the position of the functional component in the captured image displayed on the display 14 based on the position / orientation information of the information terminal 10 detected by the position detection sensor 15 and the attitude detection sensor 16. As described above, the augmented reality image 51 of the selection button is displayed on the display 14.

For example, in the example shown in FIG. 9A, the name and model of each in-vehicle component (armrest, headrest, seatbelt) specified in S23 is used as the augmented reality image 51 of the selection button at the position of each in-vehicle component in the captured image. They are superimposed and displayed on the display 14. The user can select the vehicle-mounted component to which "useful information" is posted by touching the augmented reality image 51 of the selection button provided on each vehicle-mounted component on the display 14. Here, in FIG. 9A, an example in which the name and model of the in-vehicle component are used as the augmented reality image 51 of the selection button is shown, but the present invention is not limited to this, and for example, a simple mark (graphic) in which the name and model are not described is used. It may be used as an augmented reality image of the selection button.

In S25, the processing unit 11 causes the in-vehicle component to post "useful information" by the user touching any of the augmented reality images 51 of the selection button displayed on the display 14 in S23 on the display 14. Determine if selected. If the in-vehicle component is selected, the process proceeds to S26, and if the in-vehicle component is not yet selected, S25 is repeated.

In S26, the processing unit 11 displays a posting form for inputting new "useful information" on the display 14. In the posting form, for example, as shown in FIG. 9B, an input field 52a in which "useful information" is input by the user and "useful information" input in the input field are posted to the server device 20. A post button 52b is provided. Further, the posting form may be provided with a moving image button 52c for attaching a moving image at the time of posting, a cancel button 52d for canceling the posting of "useful information", and the like.

In S27, the processing unit 11 determines whether or not the posting button 52b has been touched on the display 14 by the user. When the post button 52b is touched, the process proceeds to S28, and the processing unit 11 associates the "useful information" input by the user in the input field 52a with the model of the in-vehicle component selected by the user in S25. Is transmitted to the server device 20. On the other hand, if the post button 52b is not touched, S27 is repeated. Here, in this step, when the cancel button 52d is touched on the display 14 by the user, the posting process of "useful information" may be terminated.

As described above, in the posting process of the present embodiment, the information terminal 10 specifies the in-vehicle parts included in the captured image obtained by the camera 13, and the selection button for causing the poster to select the specified in-vehicle parts. The augmented reality image 51 of the above is superimposed on the captured image and displayed. As a result, the poster can easily link the "useful information" to the in-vehicle parts and post it to the server device 20 even if he / she does not know the model of the in-vehicle parts to which the "useful information" is posted. can do.

[Registration process of useful information]
Next, in the server device 20, a process of registering (storing) new "useful information" posted from the information terminal 10 as posted information will be described. The server device 20 of the present embodiment performs a registration process of "useful information" so as to avoid duplicate posting. FIG. 10 is a flowchart showing a process performed by the server device 20 when "useful information" is posted from the information terminal 10. Each step of the flowchart shown in FIG. 10 can be performed by the processing unit 21 of the server device 20.

In S31, the processing unit 21 receives "useful information" posted from the information terminal 10 in association with the model of the in-vehicle parts and the like. In S32, the processing unit 21 determines whether or not the "useful information" received in S31 is a duplicate post. In the present embodiment, when the posting information including the same purpose as the "useful information" received in S31 is stored in the storage unit 22 of the server device 20, it is determined that the posting is a duplicate posting, but the present invention is not limited to this. When the posted information containing exactly the same contents as the "useful information" is stored in the storage unit 22, it may be determined that the post is a duplicate post.

For example, the processing unit 21 pays attention to the item of the model of the in-vehicle part to which the "useful information" received in S31 belongs, and with respect to the posted information stored in the storage unit 22 of the server device 20 in association with the item. Performs a known text matching process. Thereby, it is possible to determine whether or not the posted information including the same purpose as the "useful information" is stored in the storage unit 22 of the server device 20. As an example, the processing unit 21 extracts some words from the text document included in the "useful information" received in S31, and the posted information including the extracted words in the text document is stored in the storage unit 22. If so, it can be determined that the "useful information" received in S31 is a duplicate post. If it is determined that the "useful information" received in S31 is a duplicate post, the process proceeds to S33, and if it is determined that the "useful information" is not a duplicate post, the process proceeds to S34.

In S33, the processing unit 21 notifies the information terminal 10 that posted the "useful information" that the "useful information" is a duplicate post. On the other hand, in S34, the processing unit 21 stores the "useful information" in the storage unit 22 in association with the item of the model of the vehicle-mounted component to which the "useful information" belongs.

As described above, in the registration process of the present embodiment, the server device 20 determines whether or not the "useful information" posted by the poster is a duplicate post, and if it is a duplicate post, to that effect. Notify the poster. As a result, it is possible to prevent a plurality of posted information containing the same purpose (or the same content) from being stored in the storage unit 22 of the server device 20, so that the same information providing program described in the first embodiment has the same information provision program. It is possible to prevent a plurality of posted information including the gist from being displayed as an augmented reality image on the display 14 of the information terminal 10. That is, the user can efficiently and easily acquire (grasp) the posted information of various purposes.

<Other embodiments>
By using the information providing system according to the above embodiment, the user can easily grasp (acquire) posted information from a third party different from the manufacturer through augmented reality. That is, the user can receive information such as information that cannot be obtained from a catalog or an operation manual. For example, information on items that are suitable for the vehicle owned by the user, items that are difficult to grasp the model of parts, such as third-party items that do not have genuine products (in other words, items that are difficult to select an appropriate item). It becomes easier to acquire information. Further, for example, if a photograph of a field, a crop, or the like created by using a general-purpose product such as a small cultivator is linked as posted information, such information can be shared among users.

In addition, the posted information may include information showing how to clean the seat in an image or text document, how to arrange the seat when staying in the car, how to arrange the seat, and the like. Further, the posted information may include travel information associated with the vehicle itself (that is, the model of the vehicle). In this case, for example, it is possible to know where a vehicle of the same model is traveling. It is also possible to register an in-vehicle parts dealer or a repair service shop as the posted information.

Further, when the posted information stored in the server device 20 includes a summary, the summary of the posted information may be displayed as an augmented reality image on the display 14 of the information terminal 10. In this case, the details of the posted information are displayed on the display 14 as an augmented reality image when the user touches an operation button provided as a part of the augmented reality image, or when the user makes a specific gesture. Can be done.

<Summary of Embodiment>
1. 1. The information providing system of the above embodiment is
An information providing system (eg 100) that provides a user with information acquired from a server (eg 20) using an information terminal (eg 10) having a camera (eg 13) and a display (eg 14).
For each of the plurality of types of articles (for example, 41a and 41b), the server stores the posted information sent from the user regarding the use of each article in association with each article.
The information terminal is
A display control means (for example, 11d) for displaying the captured image obtained by the camera on the display, and
Specific means (for example, 11b) for identifying an article contained in the captured image displayed on the display, and
An acquisition means (for example, 11c) for acquiring the posting information stored in the server in association with the article specified by the specific means from the server.
Have,
The display control means superimposes an augmented reality image (for example, 50a, 50b) of the posted information acquired by the acquisition means on the captured image obtained by the camera and displays it on the display.
According to this configuration, for example, when a user wants to obtain posted information (useful information) regarding a usage mode that is not described in an operation manual or the like for a predetermined article, the user can take a picture of the article with a camera of an information terminal. For example, it is possible to easily acquire (grasp) posted information through an augmented reality image. That is, the user can expand the range of use of the article through augmented reality.

2. In the information providing system of the above embodiment
The display control means displays an augmented reality image of the posted information so as to point to an article associated with the posted information on the display.
According to this configuration, the user can easily grasp which article the posted information is for by looking at the display.

3. 3. In the information providing system of the above embodiment
The posted information is different from the operation manual of each article.
According to this configuration, the user can obtain information that is not described in the operation manual or the like for the predetermined article, so that the range of use of the article can be expanded.

4. In the information providing system of the above embodiment
The posted information is information posted to the server by a person different from the manufacturer of each article.
According to this configuration, it is possible to expand the user community for various articles, and it is possible for the user to grasp the usage mode that the manufacturer does not anticipate.

5. The posted information includes information obtained by using each article.
According to this configuration, it is possible to enrich the information about each article and expand the community of users of the article.

6. In the information providing system of the above embodiment
When new information is posted by a person different from the manufacturer of each article, the server determines whether or not the new information overlaps with the posted information already stored in the server. When there is no duplication, the new information is stored as the posted information.
According to this configuration, since it is possible to prevent a plurality of posted information having duplicated purpose or content from being stored in the server, the plurality of duplicated posted information is displayed on the display of the information terminal as an augmented reality image. It becomes possible to prevent this.

7. In the information providing system of the above embodiment
The server notifies the person who posted the new information when there is the duplication.
According to this configuration, the poster can know that the posted information has already been posted to the server.

8. The information terminal of the above embodiment
An information terminal (for example, 10) having a camera (for example, 13) and a display (for example, 14) and providing information acquired from a server (for example, 20) to a user.
For each of the plurality of types of articles (for example, 41a and 41b), the server stores the posted information sent from the user regarding the use of each article in association with each article.
The information terminal is
A display control means (for example, 11d) for displaying the captured image obtained by the camera on the display, and
Specific means (for example, 11b) for identifying an article contained in the captured image displayed on the display, and
An acquisition means (for example, 11c) for acquiring the posting information stored in the server in association with the article specified by the specific means from the server.
Have,
The display control means superimposes an augmented reality image (for example, 50a, 50b) of the posted information acquired by the acquisition means on the captured image obtained by the camera and displays it on the display.
According to this configuration, for example, when a user wants to obtain posted information (useful information) regarding a usage mode that is not described in an operation manual or the like for a predetermined article, the user can take a picture of the article with a camera of an information terminal. For example, it is possible to easily acquire (grasp) posted information through an augmented reality image. That is, the user can expand the range of use of the article through augmented reality.

9. The information terminal of the above embodiment
An information terminal (eg 10) that has a camera (eg 13) and a display (eg 14) and posts information to a server (eg 20).
A display control means (for example, 11d) for displaying the captured image obtained by the camera on the display, and
Specific means (for example, 11b) for identifying an article contained in the captured image displayed on the display, and
A transmission means (for example, 11, 17) for transmitting the information to the server, and
Have,
The display control means superimposes an augmented reality image (for example, 51) for the user to select an article specified by the specific means on the captured image obtained by the camera and displays it on the display.
The transmission means transmits the information to the server in association with the article selected by the user via the augmented reality image.
According to this configuration, the poster can easily link the information to the article and post it to the server without knowing the model of the article to which the information is posted.

The present invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the present invention. Therefore, in order to make the scope of the present invention public, the following claims are attached.

10: Information terminal, 11: Processing unit, 12: Storage unit, 13: Camera, 14: Display, 15: Position detection sensor, 16: Posture detection sensor, 17: Communication unit, 20: Server device, 21: Processing unit, 22: Storage unit, 23: Communication unit

Claims

An information providing system that provides users with information acquired from a server using an information terminal having a camera and a display.
For each of the plurality of types of articles, the server stores the posted information sent from the user regarding the use of each article in association with each article.
The information terminal is
A display control means for displaying the captured image obtained by the camera on the display, and
Specific means for identifying an article contained in the captured image displayed on the display, and
An acquisition means for acquiring the posting information stored in the server in association with the article specified by the specific means from the server, and
Have,
The display control means is an information providing system characterized in that an augmented reality image of the posted information acquired by the acquisition means is superimposed on the captured image acquired by the camera and displayed on the display.
The information providing system according to claim 1, wherein the display control means displays an augmented reality image of the posted information so as to point an article associated with the posted information on the display.
The information providing system according to claim 1 or 2, wherein the posted information is information different from the operation manual of each article.
The information providing system according to any one of claims 1 to 3, wherein the posted information includes information obtained by using each article.
The information providing system according to any one of claims 1 to 4, wherein the posted information is information posted to the server by a person different from the manufacturer of each article.
When new information is posted by a person different from the manufacturer of each article, the server determines whether or not the new information overlaps with the posted information already stored in the server. The information providing system according to any one of claims 1 to 5, wherein the new information is stored as the posted information when there is no duplication.
The information providing system according to claim 6, wherein the server notifies the person who posted the new information when there is a duplication.
An information terminal that has a camera and a display and provides users with information acquired from a server.
For each of the plurality of types of articles, the server stores the posted information sent from the user regarding the use of each article in association with each article.
The information terminal is
A display control means for displaying the captured image obtained by the camera on the display, and
Specific means for identifying an article contained in the captured image displayed on the display, and
An acquisition means for acquiring the posting information stored in the server in association with the article specified by the specific means from the server, and
Have,
The display control means is an information terminal characterized in that an augmented reality image of the posted information acquired by the acquisition means is superimposed on the captured image obtained by the camera and displayed on the display.
An information terminal that has a camera and a display and posts information to a server.
A display control means for displaying the captured image obtained by the camera on the display, and
Specific means for identifying an article contained in the captured image displayed on the display, and
A transmission means for transmitting the information to the server, and
Have,
The display control means superimposes an augmented reality image for the user to select an article specified by the specific means on the captured image obtained by the camera and displays it on the display.
The information terminal is an information terminal characterized in that the information is transmitted to a server in association with an article selected by the user via the augmented reality image.