JP2011199800A - Information providing apparatus, portable terminal, information providing system, information providing method, and information providing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information providing apparatus in which information about a moving object can be provided to a user, and to provide a portable terminal, information providing system, information providing method, and information providing program.SOLUTION: A portable terminal 3 includes a display unit 27 and a camera. On the display unit 27, a photographed image photographed by the camera is displayed. The display unit 27 includes the user image 9 of a user who carries the portable terminal. In the user image 9, a personal information image 10 indicating personal information of the user is displayed while being superposed. On the basis of positional information transmitted from a server, the portable terminal 3 is capable of specifying the position of the personal information image displayed while being superposed on the photographic image. When the user image is moved, the positional information transmitted from the server is also moved. The portable terminal 3 is capable of displaying the personal information image while following up the moving user image.

Description

  The present invention relates to an information providing apparatus, a portable terminal, an information providing system, an information providing method, and an information providing program. More specifically, the present invention relates to an information providing apparatus, a portable terminal, an information providing system, an information providing method, and an information providing program that provide information displayed on a portable terminal.

  2. Description of the Related Art Conventionally, an information providing system that allows a user carrying a mobile communication terminal to grasp information about a surrounding area has been proposed (for example, see Patent Document 1). In the information providing system, the current position and the traveling direction of the user carrying the mobile communication terminal are specified in the central information processing apparatus. Information about the facilities existing around the user (peripheral area information) is transmitted from the central information processing apparatus to the mobile communication terminal. The mobile communication terminal displays information received from the central information processing apparatus on the display. The user of the mobile communication terminal can easily grasp the surrounding area information suitable for the current position and the traveling direction of the mobile communication terminal by viewing the display.

JP 2003-199140 A

  However, in the information providing system described above, information related to surrounding facilities is displayed on the display of the mobile communication terminal, but information related to an object moving around the user is not displayed on the display. For this reason, there exists a problem that the user cannot grasp | ascertain the information regarding the moving target object.

  An object of the present invention is to provide an information providing apparatus, a portable terminal, an information providing system, an information providing method, and an information providing program that can provide information on a moving object to a user.

  An information providing apparatus according to a first aspect of the present invention shows information acquisition means for acquiring personal information of a user input by an input means, and a position of the user corresponding to the personal information acquired by the information acquisition means. Storage control for acquiring position information for acquiring position information, the personal information acquired by the information acquisition means, and the position information acquired by the position acquisition means in association with each user in the storage means Means, a direction acquisition means for acquiring a photographing direction that can be photographed by a first photographing means that is a photographing means included in a portable terminal carried by the user, and information stored in the storage means. Position extracting means for extracting the position information of the user to be carried; the position information extracted by the position extracting means; and the direction acquiring means. Based on the shooting direction acquired in the above, the area specifying means for specifying the area that can be shot by the first shooting means of the portable terminal, and the area that can be shot specified by the area specifying means Position selection means for selecting the position information to be selected from the position information stored in the storage means; information specifying means for specifying the personal information associated with the position information selected by the position selection means; Information transmitting means for transmitting the position information selected by the position selecting means and the personal information specified by the information specifying means to the portable terminal.

  According to the first aspect of the present invention, the information providing apparatus identifies an area that can be imaged by the first imaging means of the mobile terminal. The information providing apparatus determines whether there is another user (another user) within the imageable area. When another user exists in the area, the personal information and position information of the corresponding other user are transmitted to the mobile terminal. The portable terminal displays personal information in association with the portion corresponding to the position information in the captured image. This allows the user of the mobile terminal to recognize personal information and other users in association with each other. Since the position information stored in the storage means is acquired and updated by the position acquisition means, the information providing apparatus can display accurate position information related information (individual (personal) even when the object to which the information is added moves. Information) and the information can be transmitted to the mobile terminal and associated with each other to be provided to the user.

  Further, in the first aspect, the image processing apparatus includes an image acquisition unit that acquires a captured image of the user captured by a second imaging unit capable of capturing the user, and the position acquisition unit is acquired by the image acquisition unit. The position information of the user may be acquired based on a captured image. The information providing apparatus can specify the position information of the user by performing image processing on the captured image captured by the second imaging unit. The information providing apparatus can easily identify the user's position information by a simple method without using a large-scale apparatus such as GPS.

  In the first aspect, the image acquisition unit may start acquisition of the captured image captured by the second imaging unit when the personal information is first acquired by the information acquisition unit. . Thereby, the information providing apparatus can stop the acquisition of the photographed image from the second photographing means in a state where the personal information is not received. The information providing apparatus can reliably acquire a captured image and specify the position of the user in a state where the captured image is necessary.

  Moreover, in the first aspect, the position acquisition unit is configured to use the position information of the user existing in a predetermined area at the time when the personal information is acquired by the information acquisition unit as the position of the user who has input the personal information. It may be acquired as information. Accordingly, the information providing apparatus can easily identify the user who has input the personal information and acquire the position information. For example, by setting the vicinity area of the input device through which the user inputs personal information as a predetermined area, the information providing apparatus can reliably identify the user who has input the personal information.

  In the first aspect, the position acquisition unit may acquire the position information of a user who exists in a predetermined area continuously for a predetermined time as the position information of the user who has input the personal information. Accordingly, the information providing apparatus can accurately identify the user who has input the personal information and acquire the position information. It is possible to prevent the position information of the person who has accidentally passed through the predetermined area from being acquired as the position information of the user who has input the personal information.

  In the first aspect, the information processing device further includes first receiving means for receiving the information transmitted from the portable terminal and capable of specifying the shooting direction, and the direction obtaining means is based on the information received by the first receiving means. Then, the shooting direction may be acquired by specifying the shooting direction. As a result, the information providing apparatus can easily acquire the shooting direction. For example, by providing an electronic compass on the mobile terminal, the information providing apparatus can acquire the shooting direction with high accuracy.

  Further, in the first aspect, the image acquisition unit is the identification unit attached to the portable terminal, and the identification unit capable of specifying the imaging direction is captured by the second imaging unit. The direction acquisition means may acquire the shooting direction by analyzing the captured image captured by the identification unit. Thereby, the information providing apparatus can specify the shooting direction by analyzing the image of the captured identification unit. Even when the portable terminal does not include a special device (such as an electronic compass) for specifying the shooting direction, the information providing apparatus can easily specify the shooting direction.

  In the first aspect, the information processing apparatus further comprises second receiving means for receiving the personal information transmitted from the portable terminal, wherein the position extracting means refers to information stored in the storage means, and the second receiving means The position information associated with the personal information received in the step may be extracted as the position information of the user carrying the portable terminal. As a result, the information providing apparatus receives personal information (name, telephone number, etc.) registered in advance in the mobile terminal, and can specify position information corresponding to the received personal information as position information of the mobile terminal. Since the information providing apparatus can extract the position information from the storage unit based on the personal information of the user carrying the mobile terminal, the information providing apparatus can reliably identify the position information of the mobile terminal.

  A portable terminal according to a second aspect of the present invention includes: an information receiving unit that receives the position information and the personal information transmitted from the information providing device; and the first photographing unit that is a photographing unit capable of photographing an image. The part specifying means for specifying the portion corresponding to the position information received by the information receiving means of the photographed image taken by the first photographing means, and the part specified by the part specifying means, Display control means for superimposing the personal information received by the information receiving means and displaying the information on the display means.

  According to the second aspect of the present invention, the mobile terminal receives position information and personal information from the information providing apparatus. The portable terminal displays the personal information superimposed on the portion corresponding to the position information in the photographed image photographed by the first photographing means. In the portion where the personal information is superimposed, an image of another user who has input the personal information is located. A user can easily recognize an image of another user in the captured image and personal information in association with each other. Since the position information acquired from the information providing device is sequentially updated to the latest information, the mobile terminal accurately associates the information with the object even when the object to which the information is added moves. Can be displayed and recognized by the user.

  In the second aspect, the information processing apparatus further comprises request transmission means for transmitting a request packet for requesting transmission of the position information and the personal information to the information providing apparatus, wherein the information receiving means is configured to respond to the request packet. The personal information and the position information transmitted from the information providing apparatus may be received. Accordingly, the mobile terminal can acquire the position information and the personal information from the information providing device at the timing when the mobile terminal displays the captured image on the display unit. The mobile terminal can reliably create a captured image with personal information superimposed thereon and display it on the display means. The mobile terminal can specify the display position of the personal information based on the latest position information.

  An information providing system according to a third aspect of the present invention is an information providing system including a portable terminal and an information providing apparatus that provides information to the portable terminal, the information providing apparatus being input by an input unit. Information acquisition means for acquiring personal information of the user, position acquisition means for acquiring position information indicating the position of the user corresponding to the personal information acquired by the information acquisition means, and acquired by the information acquisition means A storage control unit that stores the personal information and the position information acquired by the position acquisition unit in a storage unit in association with each user, and a photographing unit included in a portable terminal carried by the user Direction acquisition means for acquiring a photographing direction that can be photographed by the photographing means, and information stored in the storage means, before carrying the portable terminal Based on the position extraction means for extracting the position information of the user, the position information extracted by the position extraction means, and the shooting direction acquired by the direction acquisition means, the first shooting of the portable terminal A region specifying unit that specifies a region that can be imaged by the unit, and the position information included in the imageable region specified by the region specifying unit is selected based on the position information stored in the storage unit Position selecting and determining means; information specifying means for specifying the personal information associated with the position information selected by the position selecting means; the position information selected by the position selecting means; and the information specifying means Information transmitting means for transmitting the personal information specified by the mobile terminal to the mobile terminal, the mobile terminal providing the information Information receiving means for receiving the positional information and the personal information transmitted from a device, the first photographing means that is a photographing means capable of photographing an image, and a photographed image photographed by the first photographing means, A part specifying means for specifying a portion corresponding to the position information received by the information receiving means, and a display means for superposing the personal information received by the information receiving means on a portion specified by the part specifying means Display control means for displaying on the screen.

  According to the third aspect of the present invention, the information providing apparatus identifies an area that can be imaged by the first imaging means of the mobile terminal. The information providing apparatus determines whether there is another user (another user) within the imageable area. When another user exists in the area, the personal information and position information of the corresponding other user are transmitted to the mobile terminal. The portable terminal displays personal information in association with the portion corresponding to the position information in the captured image. This allows the user of the mobile terminal to recognize personal information and other users in association with each other. Since the position information stored in the storage means is acquired and updated by the position acquisition means, the information providing apparatus can display accurate position information related information (individual (personal) even when the object to which the information is added moves. Information) and the information can be transmitted to the mobile terminal and associated with each other to be provided to the user.

  An information providing method according to a fourth aspect of the present invention shows an information acquisition step of acquiring personal information of a user input by input means, and a position of the user corresponding to the personal information acquired by the information acquisition step A storage control for storing a location acquisition step for acquiring location information, the personal information acquired in the information acquisition step, and the location information acquired in the location acquisition step in a storage unit in association with each user. A step of acquiring a shooting direction that can be shot by a first shooting unit that is a shooting unit included in the portable terminal carried by the user, and the information stored in the storage unit. A position extracting step for extracting the position information of the user to be carried; Based on the position information and the shooting direction acquired in the direction acquisition step, an area specifying step for specifying an area that can be shot by the first shooting means of the mobile terminal, and specifying in the area specifying step A position selection step for selecting the position information included in the photographable area based on the position information stored in the storage means, and the position information associated with the position information selected in the position selection step. An information specifying step for specifying the personal information; an information transmitting step for transmitting the position information selected in the position selecting step; and the personal information specified in the information specifying step to the portable terminal; It has.

  According to the fourth aspect of the present invention, the information providing apparatus specifies an area that can be imaged by the first imaging means of the mobile terminal. The information providing apparatus determines whether there is another user (another user) within the imageable area. When another user exists in the area, the personal information and position information of the corresponding other user are transmitted to the mobile terminal. The portable terminal displays personal information in association with the portion corresponding to the position information in the captured image. This allows the user of the mobile terminal to recognize personal information and other users in association with each other. Since the position information stored in the storage means is acquired and updated by the position acquisition means, the information providing apparatus can display accurate position information related information (individual (personal) even when the object to which the information is added moves. Information) and the information can be transmitted to the mobile terminal and associated with each other to be provided to the user.

  The information providing program according to the fifth aspect of the present invention can cause a computer to function as each processing means of the information providing apparatus.

It is a schematic diagram which shows the outline | summary of the information provision system 1. FIG. 4 is a diagram illustrating an example of a display image displayed on a display unit 27 of the mobile terminal 3. FIG. 2 is a block diagram showing an electrical configuration of a server 5. FIG. It is a figure which shows the user table 151. FIG. 3 is a block diagram showing an electrical configuration of the mobile terminal 3. FIG. 4 is a flowchart illustrating an outline of processing executed in the mobile terminal 3, the reception device 4, the server 5, and the camera 6. It is a figure which shows an example of the picked-up image 18 image | photographed with the camera 6. FIG. It is a flowchart which shows an input process. It is a flowchart which shows a position specific process. It is a flowchart which shows a transmission process. It is a flowchart which shows a display process. FIG. 3 is a diagram illustrating an appearance of a mobile terminal 3.

  Hereinafter, an information providing system 1 according to an embodiment of the present invention will be described with reference to the drawings. These drawings are used to explain technical features that can be adopted by the present invention. The configuration of the apparatus, the flowcharts of various processes, and the like that are described are not intended to be limited to only that, but are merely illustrative examples.

  An overview of the information providing system 1 will be described with reference to FIG. The information providing system 1 includes a mobile terminal 3, a receiving device 4, a server 5, and a camera 6. The receiving device 4, the server 5, and the camera 6 are connected to the network 7. The mobile terminal 3 is connected to the network 7 via a public telephone network. The portable terminal 3 is carried by the user 2. The portable terminal 3 includes a display unit 27 and a camera 28 (see FIG. 5). The display unit 27 is provided on the front side of the mobile terminal 3. The camera 28 is provided on the back side of the mobile terminal 3. The mobile terminal 3 can display the captured image captured by the camera 28 on the display unit 27. When the user 2 is included in the captured image displayed on the display unit 27 of the mobile terminal 3, the server 5 can transmit personal information of the user to the mobile terminal 3. The portable terminal 3 can superimpose and display the personal information received from the server 5 on the portion corresponding to the user in the captured image captured by the camera 28. Personal information is input to the accepting device 4 by the user 2. The camera 6 photographs the user 2 from above.

  An example of a display image displayed on the display unit 27 of the mobile terminal 3 will be described with reference to FIG. The display image has a form in which the personal information image 10 showing the personal information is superimposed on the captured image 8 captured by the camera 28. The captured image 8 includes an image (hereinafter referred to as “user image”) 9 of a user who carries the mobile terminal 3. The personal information image 10 includes personal information (name, affiliation, age) and a frame portion surrounding the personal information. The personal information image 10 is displayed so as to overlap the user image 9. Since the user of the portable terminal 3 who visually recognizes the display image can easily recognize the user image 9 and the personal information image 10 in association with each other, the user can easily know the personal information of each user. When the user image 9 moves in the captured image 8 (arrow 31), the personal information image 10 displayed superimposed on the user image 9 also moves following the movement of the user image 9 (arrow 32). . Accordingly, the user 2 of the mobile terminal 3 can continuously recognize the personal information of each user even after the user image has moved.

  The information associated with the user image is not limited to personal information, and may be other information. Moreover, the target object to which information is targeted is not limited to the user, and may be another target object.

  The electrical configuration of the server 5 will be described with reference to FIG. The server 5 includes a CPU 11, a communication unit 12, a ROM 13, a RAM 14, and a hard disk drive (HDD) 15. The CPU 11 controls the operation of the entire server 5. The ROM 13 stores a boot program and BIOS. The RAM 14 temporarily stores various information used in the program. The HDD 15 stores an OS and a program of the CPU 11. The HDD 15 stores a user table 151 described later. The communication unit 12 performs synchronization control when communication is performed via the network 7. The CPU 11 is electrically connected to the communication unit 12, ROM 13, RAM 14, and HDD 15.

  A user table 151, which is an example of a user table stored in the HDD 15 of the server 5, will be described with reference to FIG. In the user table 151, position information and personal information are stored in association with IDs that specify each user. The position information indicates the position of the user image in the captured image of the camera 6 by coordinates (x, y). The position information is specified based on the photographed image photographed by the camera 6 and stored in the user table 151. The personal information includes “name”, “affiliation”, “age”, and “address”. In the personal information, personal information received from the accepting device 4 is stored.

  The electrical configuration of the mobile terminal 3 will be described with reference to FIG. The portable terminal 3 includes a CPU 21, ROM 22, RAM 23, flash memory 24, communication unit 25, input unit 26, display unit 27, camera 28, and electronic compass 29. The CPU 21 controls the operation of the entire mobile terminal 3. The ROM 22 stores a boot program, a BIOS, and a CPU 21 program. The RAM 23 temporarily stores various information used in the program. The flash memory 24 stores personal information registered by the user in advance and a photographed image photographed by the camera 28. The communication unit 25 performs synchronization control when communication is performed via the network 7. The input unit 26 is a button or a touch sensor that receives an input from the user. The display unit 27 is an LCD (Liquid Crystal Display) on which at least captured images are displayed. The camera 28 is a CCD camera capable of taking an image. The electronic compass 29 includes a geomagnetic sensor and can detect the orientation of the mobile terminal 3. The CPU 21 is electrically connected to the ROM 22, RAM 23, flash memory 24, communication unit 25, input unit 26, display unit 27, camera 28, and electronic compass 29.

  With reference to FIG. 6, an outline of processing executed in the mobile terminal 3, the reception device 4, the server 5, and the camera 6 will be described. First, the user carrying the portable terminal 3 inputs his / her personal information via the accepting device 4 (S1). The accepting device 4 transmits the input personal information to the server 5 (S2). The server 5 receives the personal information transmitted from the reception device 4 (S5). The camera 6 captures a predetermined area including the user who has input the personal information (S3). The camera 6 transmits the captured image to the server 5 (S4). The camera 6 repeatedly transmits captured images to the server 5 at a constant period. The server 5 receives the captured image transmitted from the camera 6 (S6).

  FIG. 7 shows a captured image 18 that is an example of a captured image captured by the camera 6. A plurality of users 2 (2a to 2d) are included in the captured image 18. A user 2d is located in front of the reception device 4. The server 5 inputs the personal information via the receiving device 4 to the user 2d located in the predetermined area 17 near the receiving device 4 when the personal information is received from the receiving device 4 in the received captured image 18. Identified as a user. The server 5 analyzes the captured image 18 to identify a user image located in the predetermined area 17. Conventionally known pattern recognition can be used as a method for specifying a user image from a captured image. The server 5 specifies the position information (x, y) of the specified user image as the position information of the user 2d. The server 5 associates the personal information received from the accepting device 4 with the specified location information of the user 2d and stores it in the user table 151 (see FIG. 4). By repeating the above processing, the position information and personal information of all users (2a to 2d) who input personal information via the accepting device 4 are stored in the user table 151.

  The camera 6 periodically transmits captured images to the server 5. The server 5 periodically receives captured images transmitted from the camera 6. The server 5 extracts the same user image as the user image included in the previously received captured image from the currently received captured image. As a result, the server 5 tracks the position of the user. As a method for tracking the position of the user, for example, the method described in Patent Document 2 can be used. The server 5 periodically specifies the position information of the user. The position information stored in the user table 151 is updated with position information that is periodically specified.

JP 2009-55139 A

  As illustrated in FIG. 6, the mobile terminal 3 that desires to acquire personal information from the server 5 and display it on the display unit 27 transmits the personal information registered in advance by the user to the server 5. The mobile terminal 3 is information on the direction of the mobile terminal 3 detected by the electronic compass 29 and information on the direction in which the camera 28 included in the mobile terminal 3 can shoot (hereinafter, referred to as “shooting direction”). (S10). The server 5 receives the personal information and the shooting direction transmitted from the mobile terminal 3 (S7). The server 5 refers to the user table 151 and performs the following processing to select position information and personal information to be transmitted to the mobile terminal 3 from the user table 151 (S8).

  For example, in FIG. 7, it is assumed that the mobile terminal 3a transmits personal information and a shooting direction to the server 5, and the server 5 receives them. The server 5 selects position information associated with the same personal information as the personal information received from the mobile terminal 3 from the user table 151. The server 5 specifies the selected position information as the position information of the mobile terminal 3a (the specified position information is assumed to be (s, t)). Next, the server 5 specifies an area 16 (hereinafter referred to as “image-capable area”) 16 that can be imaged by the camera 28 of the mobile terminal 3a. The shootable area 16 is specified based on the position information (s, t) of the mobile terminal 3a, the shooting direction (arrow 33) received from the mobile terminal 3a, and the distance U from the position information (s, t). . An area in which the angle with respect to the shooting direction is within the range of the predetermined angle θ (°) and the distance from (s, t) is within the range of the predetermined distance U (m) is specified as the shootable area 16.

  The above parameters (θ, U) may be stored in advance in the HDD 15 of the server 5 or may be input by the user via the mobile terminal 3. The portable terminal 3 may transmit the input parameters to the server 5 when transmitting the personal information and the shooting direction to the server 5. The server 5 may specify the shootable area based on the parameter received from the mobile terminal 3.

  Subsequently, position information corresponding to the imageable area 16 is selected from the position information stored in the user table 151. In FIG. 7, since the users 2b, 2c, and 2d are located in the shootable area 16, the position information of the users 2b, 2c, and 2d is selected. Subsequently, the server 5 specifies personal information associated with each of the selected position information based on the user table 151. The specified personal information corresponds to the personal information of the users 2b, 2c, and 2d.

  As shown in FIG. 6, the position information and personal information selected as described above are transmitted to the mobile terminal 3 (S9). The portable terminal 3 receives position information and personal information (S11). The portable terminal 3 specifies a position corresponding to the position information received from the server 5 among the captured images captured by the camera 28 (for example, the captured image 8, see FIG. 2). User images of the users 2b, 2c, and 2d are located at the specified positions in the captured image. A personal information image indicating the received personal information is superimposed and displayed at the specified position (S12). As described above, the mobile terminal 3 can display the personal information image in association with the user image included in the display image.

  For example, in the example shown in FIG. 7, the location information and personal information of the users 2b, 2c, and 2d are transmitted from the server 5 to the mobile terminal 3a. The portable terminal 3a displays the received personal information in a superimposed manner at a position corresponding to the received position information among the captured images displayed on the display unit 27. The personal information image of the user 2b is displayed so as to overlap the user image of the user 2b. The personal information image of the user 2c is displayed so as to overlap the user image of the user 2c. The personal information image of the user 2d is displayed superimposed on the user image of the user 2d. Thereby, the display image shown in FIG. 2 is displayed on the display unit 27.

  In addition, the portable terminal 3 transmits personal information and an imaging | photography direction to the server 5 repeatedly with a predetermined period (S10, refer FIG. 6). The server 5 receives the personal information and the shooting direction from the portable terminal 3 (S7, see FIG. 6). Since the server 5 periodically acquires captured images from the camera 6 and updates position information, the user table 151 stores the latest position information of the mobile terminal 3. The server 5 can re-extract position information and personal information to be transmitted to the mobile terminal 3 from the user table 151 based on the latest position information whose position information has been updated (S8, see FIG. 6). The portable terminal 3 can receive the position information and the personal information re-extracted based on the latest position information from the server 5 (S11, see FIG. 6) and display them on the display unit 27 (S12, see FIG. 6). Thus, even when the user moves and the user image moves in the captured image, the mobile terminal 3 can associate the personal information image with the moved user image. A user who visually recognizes the mobile terminal 3 can easily recognize the user image and the personal information image in association with each other.

  With reference to FIG. 8 to FIG. 10, processing (input processing, position specifying processing, transmission processing) executed by the CPU 11 of the server 5 will be described. When the server 5 is powered on, each process is started and executed. Each process is appropriately switched by the OS.

  The input process will be described with reference to FIG. By monitoring the signal transmitted from the receiving device 4, it is determined whether the user has started inputting personal information via the receiving device 4 (S21). The accepting device 4 transmits a start notification signal to the server 5 when the user starts inputting personal information. If the server 5 has not received the start notification signal, the personal information input to the accepting device 4 has not started (S21: NO), and the process returns to S21. Reception of the start notification signal transmitted from the accepting device 4 is continuously monitored. When the start notification signal transmitted from the reception device 4 is received (S21: YES), the input of personal information is started via the reception device 4. The state of the camera 6 is monitored (S23). When the camera 6 is not activated (S23: NO), the camera 6 is activated and photographing is started (S25). The process proceeds to S27. If the camera 6 has already been activated (S23: YES), the process proceeds to S27. As described above, since the server 5 can start photographing with the camera 6 when the input of the personal information is started, the photographed image can be reliably acquired from the camera 6 when the photographed image is necessary.

  In S27, captured images periodically transmitted from the camera 6 are received (S27). The received captured image is analyzed. It is determined whether a user image is included in a predetermined area in the vicinity of the accepting device 4 in the captured image (S29). When the user image is not included in the predetermined area near the receiving device 4 (S29: NO), a stop notification signal for stopping the input of personal information is transmitted to the receiving device 4 (S33). In the receiving device 4, the personal information input process is stopped. The process returns to S21. On the other hand, when the user image is included in a predetermined area in the vicinity of the accepting device 4 in the captured image (S29: YES), it is determined whether the user image is included in the predetermined area continuously for a predetermined time. (S31). If the user image disappears from the predetermined area before the predetermined time has elapsed (S31: NO), a stop notification signal is transmitted to the accepting device 4 (S33), and the input process is stopped. The process returns to S21.

  When the user image is continuously included in the predetermined area for a predetermined time (S31: YES), the personal information transmitted from the accepting device 4 is received (S35). The user image located in the predetermined area when the personal information is received is specified as the user image of the user who has input the personal information. As a result, the server 5 can reliably identify the user who has input the personal information. The position information of the specified user image is specified (S37). An ID that identifies the user is assigned. The ID, position information, and position information are associated with each other and stored in the user table 151 (S39). The process returns to S21.

  The position specifying means will be described with reference to FIG. In the input process (see FIG. 8), it is determined whether the ID, position information, and personal information are stored in the user table 151 (S41). If the ID, position information, and personal information are not stored in the user table 151 (S41: NO), the process returns to S41. When the personal information is stored in the user table 151 (S41: YES), a captured image periodically transmitted from the camera 6 is received (S43). A process of specifying a user image included in the captured image and specifying its position information is executed for each ID stored in the user table 151 (S45). When the user image is specified and the position information is specified (S47: YES), the position information stored in the user table 151 is overwritten with the specified position information (S49). The process proceeds to S51. On the other hand, when the user image or the position information cannot be specified (S47: NO), the process proceeds to S51 as it is.

  In S51, it is determined whether or not an ID for which identification of the user image and position information has not been attempted remains in the user table 151 (S51). If there is remaining information for which position information is not specified (S51: YES), the process returns to S47. An unprocessed ID is selected from the user table 151, and an attempt is made to specify the user image of the user corresponding to the selected ID and its position information (S47, S49). When the process of specifying the user image and the position information is executed for all IDs stored in the user table 151 (S51: NO), the process waits for a predetermined time (S53). The predetermined time is substantially the same as the cycle in which the camera 6 transmits the captured image. After waiting for a predetermined time, the process returns to S41. As a result, the process of periodically receiving the captured image from the camera 6 and updating the position information of the user image is repeatedly executed. As described above, the server 5 periodically and repeatedly identifies the position information of the user image and stores it in the user table 151. Therefore, even when the user image moves within the captured image, the position information of the user image is updated. It can be updated to information.

  The transmission process will be described with reference to FIG. It is determined whether the personal information and the shooting direction transmitted from the portable terminal 3 have been received (S61). If the personal information and the shooting direction have not been received (S61: NO), the process returns to S61. When the personal information and the shooting direction are received (S61: YES), the received personal information is acquired (S63). The received shooting direction is acquired (S65). The personal information stored in the user table 151 is searched, and it is determined whether the same personal information as the acquired personal information is stored (S67). Whether or not the same personal information is stored may be determined by determining whether only specific information (for example, name) is the same among a plurality of pieces of personal information, You may determine whether they match. If the same personal information as the received personal information is not stored in the user table 151 (S69: NO), the process returns to S61.

  When the same personal information as the received personal information is stored in the user table 151 (S69: YES), the position information associated with the corresponding personal information is selected (S71). Based on the selected position information and the acquired shooting direction, the shootable area of the mobile terminal 3 that has transmitted the personal information and the shooting direction is specified (S73). Position information included in the specified image-capable area is selected from the user table 151 (S75). Personal information corresponding to the selected position information is specified from the user table 151 (S77). The position information selected in S71 and the personal information specified in S77 are transmitted to the mobile terminal 3 that transmitted the personal information and the shooting direction (S79). The process returns to S61.

  A display process executed by the CPU 21 of the mobile terminal 3 will be described with reference to FIG. The display process is activated and executed by the CPU 21 when an application for displaying personal information superimposed on a captured image is activated.

  When the display process is activated, shooting by the camera 28 is started (S81). The personal information stored in advance in the flash memory 24 is read out. As the personal information, all of a plurality of personal information stored in the user table 151 of the server 5 may be stored, or only specific personal information may be stored. The shooting direction is acquired based on the direction information detected by the electronic compass 29 (S82). The personal information and the shooting direction are transmitted to the server 5 (S83). Thus, the portable terminal 3 can specify the shooting direction with high accuracy by using the electronic compass 29.

  In accordance with the transmitted personal information and the shooting direction, reception of position information and personal information returned from the server 5 is monitored (S85). If position information and personal information are not received (S85: NO), the process returns to S85. When the position information and the personal information are received (S85: YES), a portion corresponding to the received position information is specified in the captured image captured by the camera 28 (S87). A user image is located at a specified portion of the captured image. A personal information image indicating the received personal information is superimposed on the identified portion. As a result, a display image in which the personal information image is superimposed on the photographed image is displayed on the display unit 27 (S89). Accordingly, the personal information image is associated with the user image.

  It is determined whether an instruction to end the running application has been input (S91). When an instruction to end the application is input (S91: YES), the display process ends. When the instruction to end the application is not input (S91: NO), the process waits for a predetermined time (S93). After waiting for a predetermined time, the process returns to S82. The shooting direction is newly acquired (S82), and the personal information and the shooting direction are repeatedly transmitted to the server 5.

  As described above, the server 5 identifies the shootable area of the mobile terminal 3. The server 5 determines whether another user exists within the imageable area. If another user is present in the shootable area, the server 5 transmits the personal information of the corresponding other user to the portable terminal 3 together with the position information. The portable terminal 3 displays personal information in a superimposed manner at a position corresponding to the received position information in the captured image. Thus, the server 5 can provide the mobile terminal 3 with information (personal information or the like) associated with the position information of the moving object (person or the like). The mobile terminal 3 can display the moving object and the related information in association with each other. The user can easily recognize the moving object and the related information in association with each other.

  The server 5 can identify the position information of the user carrying the mobile terminal 3 by performing image processing on the captured image captured by the camera 6. The server 5 can easily identify the user's position information by a simple method without using GPS or the like.

  In addition, this invention is not limited to the said embodiment, A various change is possible. In the above description, the server 5 specifies the user image located in the predetermined area when the personal information is received from the reception device 4 as the user image of the user who has input the personal information. The present invention is not limited to this. For example, the server 5 may specify a user image that is located in a predetermined area continuously for a predetermined time or more as a user image of a user who has input personal information. Accordingly, for example, it is possible to prevent a user image of a person who has accidentally passed through a predetermined area from being specified as a user image of a user who has input personal information.

  In the above description, the portable terminal 3 transmits the shooting direction determined based on the direction detected by the electronic compass 29 to the server 5. The server 5 has received the shooting direction transmitted from the mobile terminal 3 and has specified a shootable area based on the received shooting direction. The present invention is not limited to this. The shooting direction may be specified by another method. For example, as illustrated in FIG. 12, an arrow-shaped identification unit 35 may be attached to the side end surface of the mobile terminal 3. The server 5 may specify the identification unit 35 from the captured image captured by the camera 6. The server 5 may specify the direction of the arrow in the identified identification unit 35 as the shooting direction. Thereby, even if the portable terminal 3 does not include the electronic compass 29, the server 5 can easily specify the shooting direction.

  In the above description, the server 5 receives the personal information from the mobile terminal 3 and selects the same personal information as the received personal information from the user table 151, thereby specifying the location information of the mobile terminal 3 that transmitted the personal information. It was. The present invention is not limited to this. The mobile terminal 3 may transmit the ID to the server 5. The server 5 may specify the position information of the mobile terminal 3 by selecting the position information associated with the same ID as the received ID from the user table 151.

  In the above description, personal information input via the accepting device 4 is stored in the user table 151. The present invention is not limited to this. The server 5 may store the personal information received from the mobile terminal 3 in the user table 151.

  In the above description, the mobile terminal 3 periodically receives the personal information and the shooting direction from the server 5 by receiving the positional information and the personal information from the server 5. The present invention is not limited to this. The server 5 may transmit the position information and the personal information to the mobile terminal 3 when any of the position information stored in the user table 151 is updated. Accordingly, the mobile terminal 3 can receive the latest position information and personal information from the server 5 without repeatedly transmitting the personal information and the shooting direction to the server 5. The mobile terminal 3 may transmit the shooting direction to the server 5 only when the shooting direction changes with movement.

  In the above description, the camera 6 periodically transmits captured images to the server 5, but the present invention is not limited to this. For example, the camera 6 may transmit the captured image to the server 5 in the following procedure. The portable terminal 3 transmits a notification signal to the server 5 when it moves. When the server 5 receives the notification signal from the mobile terminal 3, the server 5 instructs the camera 6 to perform shooting. When instructed by the server 5, the camera 6 captures a captured image and transmits it to the server 5. The server 5 specifies the position information of the user based on the received captured image. As a result, the server 5 can acquire a captured image and specify the position information of the user only when the user moves, so the processing load on the server 5 can be reduced. Note that when the mobile end 3 is moved, the server 5 can acquire the captured image and specify the position information of the user, so that the server 5 can reliably store the latest position information in the user table 151.

  The CPU 11 that performs the process of S35 of FIG. 8 corresponds to the “information acquisition unit” of the present invention. The CPU 11 that performs the processing of S37 of FIG. 8 and S45 of FIG. 9 corresponds to the “position acquisition means” of the present invention. The CPU 11 that performs the processing of S39 of FIG. 8 and S49 of FIG. 9 corresponds to the “storage control means” of the present invention. The CPU 11 that performs the process of S65 in FIG. 10 corresponds to the “direction acquisition unit” of the present invention, the CPU 11 that performs the process of S71 corresponds to the “position extraction unit” of the present invention, and the CPU 11 that performs the process of S73 of the present invention. The CPU 11 that performs the process of S75 corresponds to the “position selecting unit” of the present invention, and the CPU 11 that performs the process of S77 corresponds to the “information specifying unit” of the present invention. The CPU 11 that performs the processing corresponds to the “information transmitting unit” of the present invention. The CPU 11 that performs the processing of S27 of FIG. 8 and S43 of FIG. 9 corresponds to the “image acquisition unit” of the present invention. The CPU 11 that performs the process of S61 in FIG. 10 corresponds to the “first receiving means” and “second receiving means” of the present invention. The CPU 21 that performs the process of S85 in FIG. 11 corresponds to the “information receiving unit” of the present invention, the CPU 21 that performs the process of S87 corresponds to the “part identification unit” of the present invention, and the CPU 21 that performs the process of S89 is the present invention. The CPU 21 that performs the process of S83 in FIG. 11 corresponds to the “request transmission unit” of the present invention. The HDD 15 in FIG. 3 that stores the user table 151 corresponds to the “storage unit” of the present invention. The camera 28 in FIG. 5 corresponds to the “first photographing means” of the present invention. The camera 6 in FIG. 1 corresponds to the “second photographing unit” of the present invention.

  The process of S35 in FIG. 8 corresponds to the “information acquisition unit” of the present invention. The process of S37 of FIG. 8 and S45 of FIG. 9 corresponds to the “position acquisition unit” of the present invention. The processing of S39 of FIG. 8 and S49 of FIG. 9 corresponds to the “storage control means” of the present invention. The process of S65 in FIG. 10 corresponds to the “direction acquiring unit” of the present invention, the process of S71 corresponds to the “position extracting unit” of the present invention, and the process of S73 corresponds to the “region specifying unit” of the present invention. The process of S75 corresponds to the “position selection unit” of the present invention, the process of S77 corresponds to the “information specifying unit” of the present invention, and the process of S79 corresponds to the “information transmission unit” of the present invention.

DESCRIPTION OF SYMBOLS 1 Information provision system 3 Portable terminal 4 Reception apparatus 5 Server 6 Cameras 11 and 21 CPU
15 HDD
16 Shootable area 17 Predetermined area 24 Flash memory 26 Electronic compass 27 Display unit 28 Camera 29 Electronic compass 151 User table

Claims (13)

Information acquisition means for acquiring personal information of the user input by the input means;
Position acquisition means for acquiring position information indicating the position of the user corresponding to the personal information acquired by the information acquisition means;
Storage control means for storing the personal information acquired by the information acquisition means and the position information acquired by the position acquisition means in a storage means in association with each user;
Direction obtaining means for obtaining a photographing direction that can be photographed by a first photographing means that is a photographing means provided in a portable terminal carried by the user;
Position extraction means for extracting the position information of the user carrying the portable terminal with reference to information stored in the storage means;
Area specifying means for specifying an area that can be imaged by the first imaging means of the mobile terminal based on the position information extracted by the position extracting means and the imaging direction acquired by the direction acquiring means; ,
Position selection means for selecting, from the position information stored in the storage means, the position information included in the imageable area specified by the area specifying means;
Information specifying means for specifying the personal information associated with the position information selected by the position selecting means;
An information providing apparatus comprising: information transmitting means for transmitting the position information selected by the position selecting means and the personal information specified by the information specifying means to the portable terminal. . Image acquisition means for acquiring a captured image of the user imaged by a second imaging means capable of imaging the user;
The position acquisition means includes
The information providing apparatus according to claim 1, wherein the position information of the user is acquired based on the captured image acquired by the image acquisition unit. The image acquisition means includes
The information providing apparatus according to claim 2, wherein when the personal information is first acquired by the information acquisition unit, acquisition of the captured image captured by the second imaging unit is started. The position acquisition means includes
3. The position information of a user existing in a predetermined area at the time when the personal information is acquired by the information acquisition means is acquired as the position information of a user who has input the personal information. 3. The information providing device according to 3. The position acquisition means includes
The information provision according to any one of claims 2 to 4, wherein the position information of a user who exists in a predetermined area continuously for a predetermined time is acquired as the position information of the user who has input the personal information. apparatus. First receiving means for receiving information transmitted from the mobile terminal and capable of specifying the shooting direction,
The direction acquisition means includes
The information providing apparatus according to claim 1, wherein the shooting direction is acquired by specifying the shooting direction based on information received by the first receiving unit. The image acquisition means includes
An identification unit attached to the portable terminal, the identification unit capable of specifying the shooting direction acquires the captured image when the second imaging unit captures the image;
The direction acquisition means includes
The information providing apparatus according to claim 2, wherein the photographing direction is acquired by analyzing the photographed image photographed by the identification unit. Comprising second receiving means for receiving the personal information transmitted from the portable terminal;
The position extracting means includes
The information stored in the storage means is referred to, and the position information associated with the personal information received by the second receiving means is extracted as the position information of the user carrying the portable terminal. The information providing apparatus according to claim 1. A portable terminal that receives information from the information providing device according to claim 1 and displays the information on a display unit,
Information receiving means for receiving the location information and the personal information transmitted from the information providing device;
The first photographing means which is a photographing means capable of photographing an image;
A part specifying unit for specifying a part corresponding to the position information received by the information receiving unit out of the captured image shot by the first shooting unit;
A portable terminal comprising: a display control unit that superimposes the personal information received by the information receiving unit on a portion specified by the part specifying unit and displays the personal information on a display unit. Request transmission means for transmitting a request packet for requesting transmission of the position information and the personal information to the information providing apparatus;
The information receiving means includes
The mobile terminal according to claim 9, wherein the personal information and the position information transmitted from the information providing apparatus in response to the request packet are received. An information providing system including a portable terminal and an information providing apparatus that provides information to the portable terminal,
The information providing device includes:
Information acquisition means for acquiring personal information of the user input by the input means;
Position acquisition means for acquiring position information indicating the position of the user corresponding to the personal information acquired by the information acquisition means;
Storage control means for storing the personal information acquired by the information acquisition means and the position information acquired by the position acquisition means in a storage means in association with each user;
Direction obtaining means for obtaining a photographing direction that can be photographed by a first photographing means that is a photographing means provided in a portable terminal carried by the user;
Position extraction means for extracting the position information of the user carrying the portable terminal with reference to information stored in the storage means;
Area specifying means for specifying an area that can be imaged by the first imaging means of the mobile terminal based on the position information extracted by the position extracting means and the imaging direction acquired by the direction acquiring means; ,
Position selection and determination means for selecting the position information included in the imageable area specified by the area specifying means based on the position information stored in the storage means;
Information specifying means for specifying the personal information associated with the position information selected by the position selecting means;
Information transmission means for transmitting the position information selected by the position selection means and the personal information specified by the information specifying means to the mobile terminal;
The portable terminal is
Information receiving means for receiving the location information and the personal information transmitted from the information providing device;
The first photographing means which is a photographing means capable of photographing an image;
A part specifying unit for specifying a part corresponding to the position information received by the information receiving unit out of the captured image shot by the first shooting unit;
An information providing system comprising: display control means for superimposing the personal information received by the information receiving means on a part specified by the part specifying means and displaying the information on a display means. An information acquisition step of acquiring personal information of the user input by the input means;
A position acquisition step of acquiring position information indicating a position of a user corresponding to the personal information acquired by the information acquisition step;
A storage control step of storing the personal information acquired in the information acquisition step and the location information acquired in the location acquisition step in a storage unit in association with each user;
A direction obtaining step for obtaining a photographing direction that can be photographed by a first photographing means that is a photographing means included in a portable terminal carried by the user;
A position extracting step of referring to the information stored in the storage means and extracting the position information of the user carrying the portable terminal;
An area specifying step for specifying an area that can be imaged by the first imaging means of the mobile terminal based on the position information extracted in the position extracting step and the imaging direction acquired in the direction acquiring step; ,
A position selecting step for selecting the position information included in the imageable area specified in the area specifying step based on the position information stored in the storage unit;
An information specifying step for specifying the personal information associated with the position information selected in the position selecting step;
An information providing method comprising: transmitting the position information selected in the position selecting step and the personal information specified in the information specifying step to the mobile terminal. .   An information providing program for causing a computer to function as each processing means of the information providing apparatus according to claim 1.

Images