JP2006195786A - Method, apparatus and program for retrieving image data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To take images of a person carrying a mobile terminal and to efficiently retrieve and present the image data of the person. <P>SOLUTION: A method for retrieving the image data taken by a photographing apparatus is provided. A visitor carries a mobile terminal 1010, and any position to which the mobile terminal 1010 moved and its time are stored. The photographing apparatus 1020 stores the images taken, together with the time of imaging. A retrieval means 1050 is provided for taking out and presenting the images of the visitor who carries the mobile terminal, based upon the installed position of the photographing apparatus, the position information stored in the mobile terminal, the time information stored in the mobile terminal, and the time information stored together with the image data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image capturing / managing system that captures images, stores captured images, and acquires a desired image from the stored images.

  Many techniques related to a system for photographing visitors at theme parks and amusement parks and presenting the photographed images to visitors who have become subjects have been proposed.

  Have a visitor possess a transmitter, operate the transmitter to control the photographic device, take a picture, store the identification information of the transmitter in the server together with the captured image, and transmit from the stored image A system for searching for and acquiring images of visitors using the machine identification information as a key is disclosed (see [Patent Document 1]).

  An image capturing device installed at a specific location automatically and intermittently captures images, stores the captured images in a server connected to the network in a different location from the image capturing device, and displays the stored images. A system is disclosed in which an image designated by a visitor is printed and output to a visitor who has become a subject (see [Patent Document 2]).

  Let the visitor have an electronic tag, and the shooting device installed at the designated location will detect that the electronic tag is approaching, take a picture, and store the shot image together with the tag number of the electronic tag by the server A system for searching and acquiring an image using a tag number as a key is disclosed (see [Patent Document 3]).

  A system that consists of an imaging device and a server that stores captured images, and allows a visitor to be a subject to have a portable terminal, and the imaging device detects when the portable terminal is approaching, A system for photographing according to an instruction is disclosed (see [Patent Document 4]).

  The imaging device acquires an identification signal transmitted from an electronic device possessed by a visitor, stores the identification information of an electronic device that is present in the vicinity at the time of shooting together with the captured image, and uses the identification information of the electronic device as a key image Has been disclosed (see [Patent Document 5]).

  Let the visitor hold the mobile terminal, notify the mobile device that there is a shooting device nearby when the mobile device approaches the shooting device, and take a picture when the shooting operation is instructed by the mobile terminal A system is disclosed that stores an image together with identification information held by a portable terminal and searches for and acquires an image using the identification information as a key (see [Patent Document 6]).

A system that acquires position information and shooting time of a shooting location when a camera that is a shooting device moves, stores position information and time information together with image data in a server, and creates an electronic album from the time and position information Is disclosed (see [Patent Document 7]).
JP-A-10-224773 JP-A-11-136607 JP 2002-41703 A JP 2002-51281 A Japanese Patent Laid-Open No. 2002-112074 JP 2003-78804 A JP 2003-183742 A

  However, the conventional techniques have the following problems.

(1) Difficult to search for images In the system disclosed in Japanese Patent Application Laid-Open No. 11-136607, all captured images are presented to visitors. For example, if it is assumed that images are taken for 6 hours at 1 minute intervals, 360 images must be viewed only by the image capturing device. There is a problem that it is almost impossible to find an image of visitors in a theme park where dozens of photographing devices exist.

(2) A system in which identification information of a mobile terminal held by a visitor and an image cannot be dealt with by linking and remembering, especially a system in which the mobile terminal and the image have a one-to-one relationship (Japanese Patent Laid-Open No. 10-224773) In JP-A-2002-41703, JP-A-2002-51281, and JP-A-2003-78804, an image in which a plurality of visitors holding a portable terminal is captured is stored in association with the image. There is a problem that a visitor holding a portable terminal having a number different from the identification number is unable to retrieve the image even though he / she is shown.

(3) There is a possibility that an image in which the subject is not photographed may be retrieved. The problem of the above (2) is that the system stores an identification number of a plurality of portable terminals existing in the vicinity of the image (Japanese Patent Laid-Open No. 2002-112074). ) Can be avoided, but because it is stored with the image just because it was in the vicinity, there is an obstacle such as a tree on the back of the camera or between the camera Even if it is not captured, there is a problem that it may be searched as being captured.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to photograph a portable terminal carrier and efficiently search and present image data taken by the portable terminal carrier.

  In order to solve the above problems, an image data search method according to the present invention comprises the following arrangement. That is, a method for searching image data captured by an imaging device, the recording step for recording position information of each identifiable subject, the holding step for holding image data obtained by photographing the subject in a holding unit, A retrieval step of retrieving image data held in the holding unit that satisfies a condition based on the recorded position information of the subject and the position information of the photographing apparatus.

  In order to solve the above problems, an image data search apparatus according to the present invention comprises the following arrangement. That is, an apparatus for searching image data captured by an image capturing apparatus, recording means for recording position information of each identifiable subject, holding means for storing image data capturing the subject, and the recorded Retrieval means for retrieving image data held in the holding means that satisfies a condition based on the position information of the subject and the position information of the photographing apparatus.

  In order to solve the above problems, a program according to the present invention comprises the following arrangement. That is, a program for searching image data captured by an image capturing apparatus, a recording procedure for recording position information of each identifiable subject, a holding procedure for holding image data obtained by capturing the subject in a holding unit, A search procedure for searching for image data held in the holding unit that satisfies a condition based on the recorded position information of the subject and the position information of the photographing apparatus.

As described above, according to the present invention, since the image is searched on the condition of the position of the mobile terminal, the number of image data that is displayed and actually viewed can be narrowed down.
In addition, since the search can be performed for each portable terminal, even if the image shows a plurality of visitors who carry the portable terminal, the image data showing the visitors is presented for each visitor without excess or deficiency. be able to.

  Furthermore, by taking into account the orientation of the photographing apparatus, it is possible to accurately retrieve image data that is shown.

  In addition, the amount of information to be stored can be reduced by limiting the acquisition of position information or capturing of image data when necessary.

<Embodiment 1>
A mode in which the present invention is implemented will be described as a first embodiment.

  FIG. 1 is a diagram illustrating the overall configuration of the first embodiment. In the figure, 1010 is a portable terminal that is carried by a visitor and receives radio waves from GPS satellites to acquire the existing position, 1020 is a photographing device that captures an image, 1030 is a network that transmits image data, and 1040 is image data. An image server 1050 for storing / accumulating is a retrieval device for retrieving and displaying an image.

  FIG. 2 is a configuration diagram of the mobile terminal 1010. In the figure, 2010 is a GPS receiving unit that receives radio waves from GPS satellites and acquires position information, 2020 is a control unit that controls the entire mobile terminal 1010, and 2030 is for storing position information received by the GPS receiving unit 2010. 2040 is a search device connection unit for connecting the mobile terminal 1010 and the search device 1050, and 2050 is a bus for connecting each unit in the mobile terminal 1010.

  FIG. 3 is a configuration diagram of the photographing apparatus 1020. In the figure, reference numeral 3010 denotes an image capturing unit that captures an image and extracts image data as electronic data, 3020 a control unit that controls the entire image capturing apparatus 1020, 3030 a network unit for connecting to the network 1030 for data communication, and 3050 This is a bus for joining the respective units in the photographing apparatus 1020.

  FIG. 4 is a configuration diagram of the image server 1040. In the figure, reference numeral 4010 denotes a storage unit for storing image data sent from the photographing apparatus 1020 via the network 1030 in a database, 4020 denotes a control unit for controlling the entire image server, and 4030. Is a network unit for connecting to a network and receiving image data, and 4040 is a bus for connecting each unit in the image server 1040.

  FIG. 5 is a configuration diagram of the search device 1050. In the figure, 5010 is a mobile terminal connection unit for connecting the mobile terminal 1010, 5020 is a control unit for controlling the entire search device 1050, and 5030 is connected to the network 1030 to acquire image data from the image server 1040. A network unit, 5040 is a display unit for displaying image data and showing it to the user, 5050 is an input unit for the user to operate, 5060 is a printing unit for printing out the image data, and 5070 is in the search device 1050. This is a bus for connecting the various parts.

  In this embodiment, GPS (Global Positioning System) is used to acquire the location information of the mobile terminal 1010. However, several transmitters that transmit the location information are installed in the theme park. Other techniques for obtaining position information, such as receiving radio waves from them and specifying the position, may be used. Also, in this embodiment, the description will be made using USB (Universal Serial Bus) to connect the mobile terminal 1010 and the search device 1050, but it goes without saying that it can be replaced by another method.

  Hereinafter, the processing of this embodiment will be described with reference to the drawings. It is assumed that the system implementing this embodiment is used in a theme park.

  First, a portable terminal 1010 is lent to a visitor at the entrance of the theme park. When lending, the mobile terminal 1010 is switched on and the mobile terminal 1010 starts operating.

  When operating, the control unit 2020 is initialized, and a position storage routine in the control unit 2020 is activated at regular intervals of about 1 minute by a timer existing in the control unit 2020. In the position storage routine, the time information acquired simultaneously with the position information (latitude, longitude) received by the GPS receiving unit 2010 is stored in the storage unit 2030 in a table format as shown in FIG. The index in this table is incremented by one each time the location store routine is called.

  The photographing apparatus 1020 is installed at a predetermined location in the theme park and is facing a predetermined direction, and is switched on and starts to operate at the same time as the park opens. When the photographing apparatus 1020 is activated, the control unit 3020 is initialized, and a photographing routine in the control unit 3020 is called at a constant interval of about one minute by a timer existing in the control unit 3020. In the photographing routine, the front surface of the photographing apparatus 1020 is photographed using the photographing unit 3010. The photographed data is sent as electronic data from the photographing unit 3010 to the network unit, and the time information at the time of photographing acquired by the control unit 3020 and the camera ID given in advance for each photographing apparatus are added to the data via the network 1030. To the image server 1040.

When the image server 1040 receives image data from the photographing apparatus 1020 via the network unit 4030, the image server 1040 separates the image data, time data, and camera ID from the received data, and is a database generated in the following SQL that exists in the storage unit 4010. Store in the table. Here, camera_id, time, and photo are a camera ID, time data, and image data, respectively.
CREATE TABLE photos
(
camera_id INTERGER NOT NULL,
time TIME NOT NULL,
photo BLOB NOT NULL
);
All the imaging devices 1020 existing in the theme park have points (X-coordinate, point on the plane coordinate) where the existing position is the center of the theme park as shown in FIG. 7 and the north is the positive direction of the Y-axis. Storage unit of the image server 1040 together with the camera ID assigned to the imaging device 1020, with the horizontal direction with the direction to which the imaging device 1020 is facing as the east (positive direction of the X axis) being 0 degrees. 4010 is stored in a database table generated by the following SQL. Here, camera_id, x_pos, y_pos, and horizontal_angle are a camera ID, an X coordinate, a Y coordinate, and a camera direction, respectively.
CREATE TABLE CAMERAS
(
camera_id INTERGER PRIMARY KEY,
x_pos FLOAT NOT NULL,
y_pos FLOAT NOT NULL,
horizontal_angle FLOAT NOT NULL
);
A visitor possesses the portable terminal 1010, walks around the theme park, and passes in front of the photographing apparatus 1020. At that time, if the photographing apparatus 1020 just performs the photographing process, the visitor is photographed. When the visitor wants to see the image taken in this way, the search device 1050 is used.

  The mobile terminal 1010 is connected to the mobile terminal connection unit 5010 of the search device 1050 using the search device connection unit 2040 of the mobile terminal 1010. Since the mobile terminal connection unit 5010 and the search device connection unit 2040 are configured using USB, the control unit 5020 can detect that the mobile terminal 1010 is connected to the search device 1050. When the control unit 5020 detects that the mobile terminal 1010 is connected, the control unit 5020 sends a command to the mobile terminal connection unit 5010 to acquire the position information table shown in FIG. 6 from the storage unit 2030 of the mobile terminal 1010. The acquired position information table is sent as an image acquisition request to the image server 1040 via the network unit 5030.

  The image server to which the position information table is sent as an image acquisition request from the search device 1050 by the network unit 4030 performs processing in accordance with the flowchart shown in FIG. 8, and stores an image in which a visitor may be reflected in the storage unit 4010. Select from image data stored in.

  The processing will be described using a flowchart.

  In step S8010, position information on latitude and longitude is extracted from the position information table. The latitude and longitude are converted into the plane coordinate system of FIG. 7, and the X coordinate and Y coordinate are set to Xm and Ym, respectively.

In step S8020, an inquiry is made as to whether the value of the expression shown in FIG. This inquiry is made by the following SQL statement. In this equation, Cd is a constant representing the distance within the allowable range.
SELECT camera_id FROM CAMERAS
WHERE SQRT (POWER (Xm-x_pos, 2) + POWER (Ym-y_pos, 2)) <Cd
If a table of camera IDs cannot be acquired by this SQL or there is no table element, the process proceeds to step S8050; otherwise, the process proceeds to step S8030.

In step S8030, the orientation of the imaging device 1020 having the camera ID (Cid) extracted in step S8020 is extracted in the next SQL.
SELECT horizontal_angle FROM FROM CAMERA WHERE camera_id = Cid;
From the horizontal angle obtained in this way, the direction in which the photographing apparatus 1020 is facing is known, and it is determined from the angle of view of the photographing apparatus 1020 whether or not what exists in the position information can be photographed by the photographing apparatus 1020. FIG. 9 is a diagram showing a method for making this determination. The relative angle of the position (Xm, Ym) indicated by the position information from the position (Xc, Yc) of the photographing apparatus 1020 (east, that is, the positive direction of the X axis is 0 degree) is an expression shown in FIG. (Here, θm). The absolute value of the difference between the orientation angle of the photographing apparatus obtained from the image server and this θm is calculated, and when the value is smaller than half of the angle of view, the photographing apparatus 1020 faces the position obtained by the position information. Judge that there is. If not, the process proceeds to step S8050. Otherwise, the process proceeds to step S8040.

  In step S8040, the difference between the time information stored together with the position information and the time information of the images with the same camera ID (Cid) among the images stored in the photos table of the image server database is within a predetermined range. If it is within the range, the image is taken out as a candidate.

This image data is selected by the following SQL.
SELECT photo FROM photos WHERE camera_id = Cid AND ABS (time-Tm) <Tcnst;
The obtained image data is added to the list, and the process proceeds to step S8050.

  In step S8050, it is checked whether there are any remaining elements in the position information table acquired in step S8010. When it remains, it progresses to step S8010, and when that is not right, it progresses to step S8060.

  In step S8060, the image data included in the list is returned to the search device 1050 as a return value to the search request of the search device 1050.

  The search device 1050 that has received the image obtained from the image server displays the image data obtained by the display unit 5040 and presents it to the park attendant. When the visitor selects an image displayed on the display unit 5040 using the input unit 5050 and instructs the control unit 5020 to print out the image selected by the printing unit 5060, the control unit 5020 displays the printing unit. The selected image is printed out at 5060.

  In this embodiment, the image capturing apparatus 1020 is configured to perform image capturing at a predetermined fixed interval. However, the mobile terminal 1010 transmits an identification signal having an arrival distance of several meters to several tens of meters, and the identification signal Can be configured such that the image capturing apparatus 1020 can receive the image, and can be configured not to perform image capturing when no signal is received. Conversely, the photographing apparatus 1020 transmits an identification signal and the portable terminal 1010 can receive the identification signal, and the portable terminal 1010 is configured to store the position information only when the portable terminal 1010 is in the vicinity of the photographing apparatus 1020. Is also possible. Similarly, the position of the photographing device 1020 is stored in the mobile terminal 1010 in advance, and the mobile device 1010 detects that the stored photographing device 1020 is nearby, and only when it is nearby. It can also be configured to store the position. Furthermore, it is also possible to transmit a signal indicating that photographing is performed when the photographing apparatus 1020 performs photographing, and to store the position information when the portable terminal 1010 receives the signal. In addition, the mobile terminal 1010 transmits a signal indicating the position where the mobile terminal 1010 exists, the image capturing apparatus 1020 receives the signal, the image capturing apparatus 1020 determines whether the position of the mobile terminal 1010 can be imaged, and the image can be captured. In this case, a signal indicating that photographing is to be performed can be transmitted, and the mobile terminal 1010 can store the position information in response to the signal.

  In this embodiment, the location information is stored in the mobile terminal 1010. However, the mobile terminal 1010 is connected to the network 1030, and the location information is networked together with the mobile terminal identification information for identifying the mobile terminal 1010. It can also be configured such that the position information acquired by the mobile terminal 1010 can be searched by the mobile terminal identification information when the mobile terminal 1010 is connected to the image server 1040 by the image server 1040 and stored in the image server 1040.

  In this embodiment, since the natural appearance of the subject is photographed, the photographing device 1020 does not tell the subject who is the subject to photograph when photographing, but the photographing device 1020 has a light emitting device. Alternatively, it is possible to add a sound generation device so that a visitor is notified by sound or light of shooting.

<Example 2>
The overall configuration of this embodiment is the same as that of Embodiment 1, and is the configuration shown in FIG. In the first embodiment, the photographing apparatus 1020 is fixed, and the position and direction of the photographing apparatus 1020 are not changed. However, in the present embodiment, the photographing apparatus 1020 is also moved.

  The configurations of the mobile terminal 1010, the image server 1040, and the search device 1050 are the same as those of the first embodiment shown in FIGS. 2, 4, and 5, respectively.

  FIG. 10 is a configuration diagram of a photographing apparatus 1020 used in this embodiment. In the figure, an imaging unit 3010, a control unit 3020, a network unit 3030, and a bus 3050 are the same as the corresponding configurations of the imaging device 1020 of the first embodiment. Further, 10010 receives a GPS signal and determines the position of the imaging device 1020. A GPS receiving unit for measurement, and 10020 is an attitude detection unit for detecting a direction in which the photographing apparatus 1020 is facing.

  The method for detecting the position of the mobile terminal 1010 is the same as in the first embodiment. The imaging device 1020 sends the captured image data and the position information obtained by the GPS receiver 10010 and the direction (angle with 0 degrees east) data obtained by the attitude detector 10020 to the image server.

The image server 1040 stores the image data, position, and orientation sent from the imaging device 1020 in a database table generated in the next SQL. Here, photo, time, x_pos, y_pos, and angle are image data, time, X coordinate, Y coordinate, and camera orientation, respectively.
CREATE TABLE photos (
photo BLOB NOT NULL,
time TIME NOT NULL,
x_pos FLOAT NOT NULL,
y_pos FLOAT NOT NULL,
angle FLOAT NOT NULL
);
Only processing that is different from the first embodiment will be described for processing when the portable terminal 1010 is connected to the search device 1050 to search for an image.

  In the first embodiment, the processing performed in steps S8020 to S8040 in the processing flow of FIG. 8 is performed as follows.

In the next SQL, the distance from the position of the position information is within a predetermined distance, and the image data within the predetermined range and the position and orientation of the photographing apparatus 1020 are acquired.
SELECT photo, x_pos, y_pos, angle FROM photos
WHERE (SQRT (POWER (Xm-x_pos, 2) + POWER (Ym-y_pos, 2)) <Cd)
AND (ABS (time-Tm) <Tcnst);
Of the acquired image data, it is determined whether a visitor holding the mobile terminal 1010 is taken from the direction of the photographing apparatus 1020 as in the first embodiment, and images that may be taken are registered in the list. .

  In this way, even if the photographing apparatus 1020 moves, it is possible to search for image data in which a visitor carrying the mobile terminal 1010 is captured.

  Furthermore, the layout of buildings and trees in the park is stored in the image server 1040, and it is calculated whether there is an obstacle between the photographing device 1020 and the portable terminal 1010, and the portable terminal 1010 is hidden behind the obstacle. In such a case, it is possible to judge whether or not a visitor is shown by not registering in the list.

Overall configuration diagram of Embodiment 1 Configuration diagram of the portable terminal of FIG. FIG. 1 is a block diagram of the photographing apparatus of FIG. Configuration diagram of the image server of FIG. Configuration diagram of the search device of FIG. A table that stores location information used on mobile devices Coordinate system that expresses location information in the theme park Processing flow of search processing performed on the image server Conceptual diagram of a method for checking whether the photographing device is facing the direction of the mobile terminal Configuration diagram of photographing apparatus of embodiment 2

Explanation of symbols

DESCRIPTION OF SYMBOLS 1010 Portable terminal 1020 Imaging device 1030 Network 1040 Image server 1050 Search device 2010 GPS receiving unit 2020 Portable terminal control unit 2030 Portable terminal storage unit 2040 Search device connection unit 2050 Portable terminal bus 3010 Imaging unit 3020 Imaging device control unit 3030 Imaging Device Network Unit 3050 Imaging Device Bus 4010 Image Server Storage Unit 4020 Image Server Control Unit 4030 Image Server Network Unit 4040 Image Server Bus 5010 Mobile Terminal Connection Unit 5020 Search Device Control Unit 5030 Search Device Network Unit 5040 Display unit 5050 Input unit 5060 Printing unit 5070 Retrieval device bus 10010 Imaging device GPS receiving unit 10020 Posture detection unit

Claims (10)

A method for retrieving image data taken by a photographing device,
A recording process for recording position information of each identifiable subject;
A holding step of holding image data obtained by photographing the subject in a holding unit;
An image data search method comprising: a search step of searching image data held in the holding means that satisfies a condition based on the recorded position information of the subject and position information of the photographing apparatus.   The image data search method according to claim 1, wherein the position information is recorded at an arbitrary time interval.   2. The image data search method according to claim 1, wherein in the search step, the search is performed in consideration of the photographing range information of the photographing apparatus.   The image data search method according to claim 1, wherein the position information of the subject is detected by a portable terminal carried by the subject.   2. The image data search method according to claim 1, wherein in the recording step, time information at which position information is recorded is recorded.   5. The position information of a subject carrying the portable terminal is recorded in the recording step when detecting that the portable terminal is present in a shootable range of the photographing apparatus. Image data search method.   The image data search method according to claim 6, wherein the portable terminal stores information on a photographing range of the photographing apparatus. A device for retrieving image data taken by a photographing device,
A recording means for recording position information of each identifiable subject;
Holding means for holding image data obtained by photographing the subject;
An image data search apparatus comprising: search means for searching image data held in the holding means that satisfies a condition based on the recorded position information of the subject and position information of the photographing apparatus. A program for retrieving image data taken by a photographing device,
A recording procedure for recording position information of each identifiable subject,
A holding procedure for holding image data obtained by photographing the subject in a holding unit;
A program for causing a computer to execute a search procedure for searching for image data held in the holding unit that satisfies a condition based on the recorded position information of a subject and position information of a photographing apparatus. A computer readable memory storing the program according to claim 9.

Images