JP5664729B2 - Search method, search device, program, search condition setting method, information terminal device - Google Patents

Description

  The present invention relates to a search apparatus and a method for searching for a predetermined type of data such as image data as a search target. The present invention also relates to a program executed by such a search device.

JP 2004-233150 A

For example, by combining a device having a GPS (Global Positioning System) function with an imaging device such as a video camera or a digital still camera, an image data file (image) recorded by a moving image or a still image recorded by these imaging devices. As the additional information of (file), it becomes possible to add position information corresponding to the shooting time.
The image file to which the position information is added in this way can be used by, for example, map application software. In other words, when an image file with location information added is read into the map application software, the map application software uses the map image to indicate the position at which the image file was shot (recorded) along with the image of the read image file. As shown.

  If position information is added to image data as described above, convenience such as easy cooperation with a map image can be obtained. However, at present, the position information added to the image data in this way is only applied to the extent that the map application uses it. Accordingly, an object of the present invention is to enable more effective use of image data to which position information is added.

Therefore, the following procedure is executed as a search method of the search device according to the present invention.
That is, a search method executed by a search device that can communicate with an information terminal, the search region range determining procedure for determining a search region range based on character string information received as a search condition from the information terminal, and the search Map information acquisition procedure for acquiring map information based on the search region range determined by the region range determination procedure, and image in which position information is associated based on the search region range determined by the search region range determination procedure Image data acquisition procedure for acquiring data, map information acquired by the map information acquisition procedure, and display for displaying the image data acquired by the image data acquisition procedure on the display means of the information terminal A display data generation procedure for generating data, and a transmission procedure for transmitting the display data to the information terminal. After the data has been sent, when receiving the search area range information from the information terminal, the search area range information received from the information terminal is displayed on the information terminal based on the display data Information corresponding to the coordinates of the map image, information corresponding to the coordinates of the map image is converted into position information, and the converted search area range information is used in the search area range determination procedure; I decided to do it.

In the present invention, the search device is configured as follows.
Based on the character string information received as the search condition from the information terminal, the search area range determining means for determining the search area range, and the map information is acquired based on the search area range determined by the search area range determining means. Map information acquisition means, image data acquisition means for acquiring image data associated with position information based on the search area range determined by the search area range determination means, and map acquired by the map information acquisition means Information and display data generation means for generating display data for displaying the image data acquired by the image data acquisition means on the display means of the information terminal, and the display data to the information terminal A transmission means for transmitting, and when the search area range information is received from the information terminal after the display data is transmitted. Search area range information received from the information terminal is the information corresponding to the coordinates for the map image displayed on the information terminal based on the display data, information corresponding to the coordinates for the map image Is converted into position information, and the converted search area range information is used as a search area range used by the search area range determination means .

  As described above, in the present invention, when a search is performed using a character string, information and data in a predetermined format related to, for example, a region or a location indicated by the character string has been retrieved as a search result. It becomes possible to obtain a search system. In addition, as described above, the search device according to the present invention is based on the premise that position information is added to the search target information. This includes, for example, image data and the like. This leads to further effective utilization of data having a structure including position information as additional information.

It is a figure which shows the example of the network system structure which consists of the search server of this Embodiment, and PC using this search server. It is a figure which shows the example of a search screen of embodiment displayed on a web browser. It is a figure which shows the example of a search screen of embodiment displayed on a web browser. It is a figure which shows the structural example of the personal computer (PC) corresponding to embodiment. It is a figure which shows the structural example of the search server corresponding to embodiment. It is a figure which shows the example of a structure of the image file corresponding to embodiment. It is a figure which shows the example of a process sequence between PC and a search server according to operation | movement of the search screen display of FIG. It is a figure which shows the example of a process sequence between PC and a search server according to operation | movement of the search screen display of FIG. It is a figure which shows typically the example of a model for the search server of embodiment to collect an image file. It is a figure which shows the structural example of the imaging device shown by FIG.

Hereinafter, modes for carrying out the present invention (hereinafter referred to as embodiments) will be described.
In this embodiment, the configuration of the search device of the present invention is applied to a search server placed on a network such as the Internet.
FIG. 1 simply shows a system configuration including a search server 3 of the present embodiment and a personal computer (PC) 2 that is a network terminal using the search server 3. First, an outline of a search method using the search server 3 of the present embodiment will be described with reference to FIG.
For example, when a user of the PC 2 tries to perform a search using the search server 3, the Web browser 34 a is activated on the PC 2. The web browser 34a is one of application software, and is for accessing and browsing web contents (web pages) published on the Internet. Then, the user performs an operation on the activated Web browser 34a, accesses the search server 3 via the network, and requests a search screen. As a result, Web page data serving as a search user interface screen is transmitted from the search server 3, and the received Web page data is interpreted on the PC 2 side, and the search screen is displayed by the Web browser 34a. As is well known, a Web page is described in a markup language such as HTML (Hyper Text Markup Language) or XML (eXtensible Markup Language), and the Web browser interprets this description and Playback output is performed.

On the search screen displayed by the Web browser as described above, an input box for inputting a search condition by a character string is displayed. Therefore, the user inputs an arbitrary character string into the input box and then performs a button operation for requesting a search. In response to this operation, the Web browser 34a of the PC 2 makes a search request to the search server 3 using the character string data input in the input box as a search condition by communication via the network.
In the search server 3, as a response to the search request, a search process is executed using a character string as a search condition transmitted along with the request. Then, the contents of the search result obtained by this search process are returned to the PC 2 as data in a format that can be displayed as a Web page.
The PC 2 causes the Web browser 34a to display and output the contents of the search result transmitted as described above.
As understood from the above description, when the user uses the search server 3 of the present embodiment, the personal computer is used in the same manner as when using a search site on the Internet as known so far. It is only necessary to operate the application of the Web browser with a terminal such as the above, and in this respect, use of a special device or difficult operation is not required.

Next, an example of the content of the search service provided by the search server 3 of the present embodiment will be described with reference to FIGS. 2 and 3 show examples of display modes of a search screen displayed on the display of the PC when the search server 3 is accessed by the PC 2 and the search service is used.
First, as described with reference to FIG. 1, the user of the PC 2 who intends to use the search service of the search server 3 starts a Web browser on the PC 2 and accesses the search server 3 using this Web browser. Then, a search screen in which an input box for inputting search conditions is arranged is displayed on the display of the PC 2 as an initial screen.

The search service of the search server 3 according to the present embodiment is an image search. Here, in order to simplify the description, it is assumed that the image to be searched is a photographic image (photo image data file).
The user in this case wants to search for a photographic image related to Mt. Fuji. Therefore, the user inputs the character string “Mt. Fuji” in the input box of the search screen displayed on the Web browser by, for example, a character input operation using a keyboard or the like, and then performs a button click operation for executing the search, for example. The operation corresponding to is performed. In response to this operation, the search request is transmitted from the Web browser to the search server 3 after designating the character string information “Mt. Fuji” as a search condition.

  The search server 3 executes a search in response to the search request and returns the search result to the Web browser of the PC 2. This search result is displayed in the window of the Web browser, for example, as shown in FIG. In FIG. 2A, a list of photo images as a search result determined to correspond to the character string information of “Mt. Fuji” as a search condition is displayed in an array (search image list display).

  In addition, in the window of the Web browser, the search area range display shown in FIG. 2B is displayed together with the list display of the search result images shown in FIG. As the search area range display, first, a map image of a certain area range is displayed. A reference position rp is displayed for a certain point in the map image, and a search range designation frame MR having a predetermined shape that is positioned with reference to the reference position rp is superimposed on the map image. It has become. The search range designation frame MR in this case is a rectangle corresponding to the image frame shape of the search area range display, for example.

  In the search area range display in FIG. 2B, a map around Mt. Fuji is displayed. The reference position rp is set to the summit of Mt. Fuji, and an area surrounded by a certain range of rectangles is indicated by a search range designation frame MR based on the reference position rp. The search range designation frame MR displayed in this way indicates “a range of positions corresponding to images listed in the search image list display of FIG. 2A”. In other words, it indicates that “the photograph images listed in the search image list display of FIG. 2A are actually taken in the area indicated in the search range designation frame MR”. Yes. Therefore, the search image list display of FIG. 2A is used as an output screen for the main search results, whereas the search area range display of FIG. 2B is the search image list display. The output search result has a role of assisting the main search result in the sense that the region range (range of position information) that is the substantial search condition is visually indicated. I can say that.

  The search area designation frame MR in the search area range display shown in FIG. 2B can be changed in size on the map image in response to an operation on the PC 2. As this operation, for example, an operation for changing the size of the search range specification frame MR while maintaining the aspect ratio by a drag operation with a mouse or the like like a range specification operation in application software for image processing or the like. Can think. Of course, the aspect ratio of the search range designation frame MR may be changed. Furthermore, it is also conceivable to move and change the position of the search range designation frame MR on the map image together with the reference position rp.

  As a specific example of the size change of the search range designation frame MR, it is assumed here that the search range designation frame MR shown in FIG. 2B has been resized as shown in FIG. The intention that the user has performed such an operation this time is as follows. In other words, for example, as a user wanted an image of the area around Mt. Fuji, he entered “Mt. Fuji” as a search character string and searched for it. Suppose you wanted an image near Mt. Fuji. However, Yamanakako is not included in the area range indicated by the search range designation frame MR shown in FIG. Therefore, in such a case, as a size change operation of the search range designation frame MR, the user expands the size to a size larger than before, for example, as shown in FIG. Such an area range is set.

In response to the change in the size of the search range designation frame MR in this way, the Web browser of the PC 2 retrieves information indicating the area range on the map according to the changed size of the search range designation frame MR. As a condition, a search request is made to the search server 3 again. The search server 3 performs a search according to this search condition and returns a search result. As a result, the search image list display lists photographic images taken in the region indicated by the changed search range designation frame MR as shown in FIGS. 2 (a) to 3 (b). Changed to content. That is, in the search image list display, in addition to the images of Mt. Fuji so far, an image photographed in the vicinity of Lake Yamanaka is added and displayed.
Although explanation here is omitted, a new area range setting is performed by an operation of reducing the size of the search range designation frame MR in the opposite manner to the transition from FIG. 2B to FIG. 3A. Of course it is possible to do.

  As understood from the above description, in the photo image search of the present embodiment, the first search condition is character string information. Conventionally, a search character string for searching for an image is generally searched for, for example, an image file name or text content added to an image layout on a Web page or the like. On the other hand, in the present embodiment, the information of the search character string specifies the location and area where the photographic image to be searched is taken.

  For example, if you want to search for photographic images with Mt. Fuji as in the above example, you can enter the character string “Mt. Fuji” as a search condition and search for Mt. Fuji. It is possible to retrieve photographic images. However, since the image search so far is based on, for example, the file name and the text of the Web page, for example, a file name that does not include “Mt. Fuji” is added, or the text content of the Web page is included. If there is no character string “Mt. Fuji”, it will be excluded from the search results. On the other hand, in the present embodiment, the search character string corresponds to a name such as a place where a photographic image is taken, and thus is related to the file name and the text content of the Web page as described above. Instead, it is possible to search for an image taken in a place or region designated as a search condition. In other words, in the case of performing an image search from the viewpoint of the location where the image was taken, a search result that is more accurate than the conventional image search can be obtained.

Further, in the present embodiment, the search range instruction display shown in FIG. 2B is also performed in addition to the search image list display shown in FIG. Thus, by looking at the search range designation frame MR in the search area range display, the user can determine in what area range the image presented in the search image list display is actually taken. It will be visually recognizable. Further, regarding the change of the search condition range, as shown in FIGS. 2B and 3A, the size of the search range designation frame MR on the map image is changed in the search area range display. Can be done by simple operation. For example, if you want to search for images related to “Mt. Fuji” and “Lake Yamanaka” according to the previous search method, enter the strings “Mt. Fuji” and “Lake Yamanaka” in the input box, and then enter these characters. For example, it is necessary to perform an operation for performing a logical sum operation on a column. The operation is not as sensual as the present embodiment, and the degree of difficulty is slightly higher.
The image search of the present embodiment shown in FIG. 2 and FIG. 3 includes position information as additional information for the image data file that is the search target content in the search server 3 as described later. This is realized by storing and managing image data and linking with map information (map database). In this respect, the image search according to the present embodiment is a service and technology that can provide a search system that has never existed before, after effectively using image data to which position information is added. I can say that.

Hereinafter, a technical configuration for realizing the image search of the present embodiment will be described.
First, a configuration example of a personal computer (PC) 2 used by a user for using a search service will be described with reference to FIG.
In this figure, the PC 2 connects the CPU 31, ROM 32, RAM 33, HDD (hard disk) 34, operation input processing unit 35, display output processing unit 36, media drive 37, data interface 38, and network interface 39 to each other via an internal bus 40. It is shown as consisting of

  A CPU (Central Processing Unit) 31 expands and executes, for example, a program held in the ROM 32 and an application program stored in the HDD 34 on the RAM 13. Further, the RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes. The ROM 32 generally has a non-rewritable configuration, but may be configured to include a nonvolatile rewritable memory element such as a flash memory, for example.

The HDD 34 serves as an auxiliary storage device in the PC, and is stored such that various programs executed by the CPU 31 are installed as described above. Various application files can also be stored. Data stored in the HDD 34 in this way is managed by a file system or the like. The CPU 31 can write or read data to or from the HDD 34 by using, for example, a file system.
In this figure, among the programs installed in the HDD 34, the Web browser 34a is particularly shown. The Web browser 34a is application software for displaying, outputting, browsing, and using a document in a markup language such as HTML uploaded or published on the Internet or locally. As will be understood from the above description, the correspondence with the present embodiment is used when accessing the search server 3 via the network (Internet) for image search.

The operation input processing unit 35 receives a signal corresponding to an operation output from the operation device 41, converts the signal into a command that can be processed by the CPU 31, and outputs the command to the CPU 31. The CPU 31 executes processing according to this command. The operation device 41 here is a comprehensive illustration of various operation input devices for a user to operate a computer such as a mouse and a keyboard.
The display output processing unit 36 executes a process for causing the display device 42 to display and output an image under the control of the CPU 31, for example.

  The media drive 37 is configured to be able to read / write data corresponding to a specific type of medium such as a CD-ROM, a DVD, or a semiconductor memory device according to a predetermined standard in a removable format, for example. Is done. The CPU 31 executes control for this purpose.

  The data interface 38 is a part configured by mounting hardware and software for communicating with an external device by a predetermined data communication method between the PC 2 and the external device. In this case, for example, an imaging apparatus Thus, a communicable configuration corresponding to one data interface 24 is adopted.

  The network interface 39 is a part that realizes an interface function for performing communication via a network such as the Internet or a LAN. For example, in the current state, the network interface 39 has a configuration corresponding to a standard such as Ethernet (registered trademark). A configuration corresponding to a wireless LAN (Local Area Network) standard such as IEEE802.11a / b / g is also conceivable. Furthermore, if it is configured to connect to the Internet using a telephone line, it is configured with a modem or the like.

Next, a configuration example of the search server 3 will be described with reference to FIG.
The search server 3 shown in this figure is shown as configured by connecting a control unit 50, a network interface 51, a search processing unit 52, and a storage unit 53 to each other via an internal bus 54.

  The control unit 50 includes, for example, a CPU, a RAM, a ROM, and the like, and the internal CPU performs a control process in the search server 3 by executing a program stored in the internal ROM or the storage unit 53. To be done.

  The network interface 51 is a part that realizes an interface function for performing communication via a network such as the Internet or a LAN. The search server 3 is connected to a terminal to which a service is to be provided through the network interface 51 via the Internet.

The search processing unit 52 is a part for executing a search process on the image file database 53a and the map database 53b in the storage unit 53 according to the input search conditions.
As a function of the search processing unit 52, for example, a memory that is a hardware resource that actually forms a search server 3 by the CPU in the control unit 50 executing a program corresponding to the function as the content creation processing unit 52. It is realized by using a signal processing circuit or the like.

  The storage unit 53 actually includes, for example, an HDD, and stores and holds various information and data necessary for operations to be executed by the search server 3. As such information, in the figure, an image file database 53a and a map database 53b are shown.

The image file database 53a is a database of image files serving as search target contents.
Here, FIG. 6 shows an example of the structure of an image file stored and managed in the storage unit 53 as the image file database 53a.
The structure of the image file shown in this figure is based on an image data portion consisting of the substance of image data as a still image (which may be a moving image), and information on predetermined items related to the contents as the image data portion. And a metadata part. An area for storing position information is provided at a predetermined position in the metadata portion. As described above, since the position information stored here is acquired by the GPS module, the position when the image data is captured is indicated by the position information. As information other than the position information stored in the metadata part, for example, the shooting date and time, the data (file) size, the resolution of the image data part, the data rate, the encoding format, and the information set in the imaging device at the time of shooting are set. The setting values of various parameters such as exposure and white balance.

  Further, the map database 53b in the storage unit 52 of FIG. 5 is map information stored in a database. The map information included in the map database 53b has a basic structure in which position information based on latitude / longitude is associated with a map image. In addition, for example, a place name, an address, and other various additional information are associated with the position information.

Although not shown here, in the storage unit 3, it is possible to also store programs necessary for the operation of the search server 3. For example, in practice, an OS (Operating System) program and an application program operating on the OS are stored. As an application program in this case, a program corresponding to a function such as the search processing unit 52 can be cited.

Next, FIG. 7 shows a processing procedure executed by the PC 2 and the search server 3 when performing an image search according to the example described in FIG. In the procedure shown in this figure, when the Web browser 34a is started on the PC 2 and accesses the Web search server 3 via the Internet, an input box for character string input, which is the top page of the search screen, is arranged. It is assumed that the search is started from the situation where the search screen is displayed.
In FIG. 7, the PC 2 side shows a user interface and a Web browser, and the search server 3 side shows a search process, a map database, and an image file database. The user interface on the PC 2 side is a process corresponding to the operation input processing unit 35. The Web browser of the same PC 2 is stored in the storage unit 34, and corresponds to the operation as the Web browser 34a executed by the CPU 31 in the activated state. The search processing, map database, and image file database on the search server 3 side correspond to the search processing unit 52, map database 53b, and image file database 53a in FIG. 5, respectively.

  As described above, in order to search for an image, first, the user inputs a character string as a search condition to the input box on the search screen top page displayed by the Web browser of the PC 2. Perform operations for input. In response to this operation, the PC 2 inputs character string information of the search condition from the user interface to the Web application as shown in step S101 in FIG. As a result, for example, a character string corresponding to the input operation is displayed in the input box of the search screen displayed by the Web browser. Assuming that an operation for a search request is performed after completing the input of the search character string in this way, the Web browser 34a is input to the search server 3 as shown in step S102. A search is requested with transmission of character string information as a search condition.

This search request is delivered to the search processing unit 52 in the search server 3. In response to this, the search processing unit 52 executes the procedure as Step S103. This Step S103 accesses the map database 53b and corresponds to the character string information as the search condition delivered together with the search request. To search for location information.
As described above, the map database 53b has a basic structure in which position information is associated with a map image, and is further made to correspond to the position information. It is also assumed that character string information of various names to be shown on a map such as That is, it can be said that the map information has a structure of information in which the character string information representing the place name and the like has a correspondence relationship with the position information.
Therefore, as step S103, the map-related character information corresponding to the character string information of the search condition is first specified from the map database 53b. Further, the position information of the map image associated with the map-related character information searched as described above is searched from the same map database 53b. Note that the position information searched in this way is position information of one point corresponding to a certain place or point. In this way, the search processing unit 52 acquires and acquires the position information searched by the procedure of step S103 in step S104. The position information acquired here is the reference position rp in the search area range display.

Subsequently, the search processing unit 52 performs a process for determining a search area range in step S105. This search area range corresponds to the search range designation frame MR arranged at the position determined by the reference position rp on the map image in the search area range display as shown in FIG. It is said.
The search area range can be determined as follows, for example. First, the map database 53b is configured such that character information managed in association with a map image is classified and managed for each name type. Examples of the classification include an address (the address can be further subdivided into prefectures, municipalities, and dolls), a facility name, a mountain, a river, and a historic site.
The character string information of the search condition is determined as to what type of character information is managed in the map database 53b, and the search area range is determined according to a predetermined rule according to the determination result. Is done.
For example, “Mt. Fuji” is classified as a mountain, but in the case of a character string of such classification, for example, even if a search area range of about several kilometers square is determined, for example, only the area near the top of Mt. Fuji is specified. Therefore, it is difficult to obtain an effective image file as a subsequent search result. Therefore, in the case of such classification, it is preferable that the search area range is from several kilometers to several tens of kilometers. Conversely, it is not appropriate to specify a range of several kilometers in the case of facilities, stores, etc. such as “○ × ward office” or “○ △ department store”, and it should be in the range of tens to hundreds of meters. Is preferred.
Considering this, for example, a rule for determining the search area range according to the classification of the character information is set, and the search processing unit 52 uses the algorithm according to this rule to search the search area range. May be configured so as to determine the above.
Note that the entity of the information in the search area range is based on the position information. As an example only, if the shape of the search area range is a rectangle corresponding to the search range specification frame MR in FIG. 2B, the four apex angles that form this rectangle correspond. Position information on the map image to be obtained. With these pieces of position information, position information included in the rectangular search area range can be specified. Further, for example, it may be possible to have information on the reference position rp and the length of the diagonal line determined according to the size of the search range designation frame MR as the search area range. If a predetermined calculation is performed using information on the quasi-position rp and the length of the diagonal line, it is possible to obtain position information that forms the search range designation frame MR. In accordance with this, even when the shape of the search range designation frame MR is circular, elliptical, or the like, the position information can be specified by executing a predetermined calculation using the reference position rp and the radius, for example.
Thus, the search area range determined in step S105 is based on the position information.

When the search area range is determined as described above, the search processing unit 52 searches the image file database 53a for an image file using the position information corresponding to the determined search area range as a search condition in step S106. .
As described above with reference to FIG. 6, position information is added to the image file. In step S106, an image file to which position information matching the position information corresponding to the search area range is added is searched. Then, by the procedure as the subsequent step S107, the image file data that is considered to satisfy the search condition in this way and the link information that indicates the location of these image files by the description such as URL are obtained from the image file database 53a. take in.
According to the description so far, in step S101, character string information corresponding to a place name or the like is input as a search condition. This character string information is converted into position information in steps S103 and S104, and is actually The image file database 53a is searched using this position information as a search condition. That is, the user interface side is searching based on character information, but is actually searching based on position information. This is because the search condition based on the character string information input from the user interface is provisional, and the position information for actually searching the image file database 53a on the search server 3 side is the actual search condition. You can see.

In subsequent step S108, processing for creating Web page data as a search image list display as shown in FIG. 2A is executed. In creating the Web page, the image file data captured in step S107 is used to generate a thumbnail image TN to be arranged in the search image list display. Further, the link information of the image file captured in the same step S107 is described in a document such as HTML so that a state where the thumbnail image is linked is obtained, for example.
Then, the search processing unit 52 returns the Web page data of the search image list display created as described above to the search request source PC 2 as a search result in step S109. The PC 2 interprets the received web page data by the web browser 34a and reproduces and outputs it. As a result, as shown in FIG. 2A, the search image list is displayed on the screen of the Web browser.

In addition, when the search processing unit 52 of the search server 3 transmits the Web page data of the search image list display in the above step S109, the part corresponding to the search area range is then retrieved from the map database 53b in steps S110 and S111. Map information is retrieved and imported.
As described above, the search area range determined in step S105 is made up of position information. As step S110, according to FIG. 2B, the partial map information corresponding to the area range in which the range based on the position information that is supposed to form the search area range is included at a certain ratio is searched. . The partial map information searched and acquired in this way is at least information of a map image corresponding to the searched area range.

  Then, in the next step S112, the search processing unit 52 uses the acquired partial map information to create a Web page for the search area range display illustrated in FIG. The search area range display web page may be created as content data using, for example, a document such as HTML, but the search area designation displayed on the map image is particularly relevant to the search area range display web page. Constructed so that the size of the frame MR can be changed according to the user operation, and a new search condition is set according to the size of the changed search range designation frame MR and a search request is transmitted. There is a need to. In consideration of providing such a user interface function, it is also effective to create a search area range display Web page as a program called a Java (registered trademark) applet, for example. As is well known, a Java applet is a program that is created in a programming language called Java and that is executed on a Web browser on which Java software is installed.

  The search area range display Web page data created as described above is transmitted to the PC 2 in step S113. The PC 2 receives the Web page data for the search area range display that has been transmitted and passes it to the Web browser 34a. The Web browser 34a performs processing on the Web page data, thereby displaying the search area range as shown in FIG. 2B, for example.

Subsequently, as described above with reference to FIG. 3, the search image list display is performed in response to an operation for changing the size of the search range designation frame MR on the search region range display displayed by the Web browser. A procedure for changing the displayed search result (the contents of the list of image files) will be described with reference to FIG.
In this case, first, in response to an operation for changing the size of the search range designation frame MR, a command corresponding to the operation is output from the user interface to the Web browser according to the procedure as step S201. In response to the input of this command, in the Web browser, for example, as shown as a transition example from FIG. 2B to FIG. Image processing for changing the MR size is executed.

Next, the Web browser sets a search area range according to the size of the search range designation frame MR changed in step 201 by the procedure as step S202, and sends information on this search area range to the search server 3. To send. The entity of the search area range information at this stage may be expressed based on the position information associated with the map image, for example, but here, for example, when viewed as a map image It is assumed that it is expressed by the coordinates on the image. That is, information on the search area range based on the map information is transmitted from the Web browser.
If the search area range to be set and transmitted in step S201 is based on position information, the search area range display Web page created in step S112 of FIG. 7 corresponds to the coordinates of the map image. Therefore, it is necessary to add location information, and the data size of the Web page increases, which may be disadvantageous in terms of network transmission time. In addition, if the Web browser tries to execute Web page data displaying such a search area range, the processing becomes heavy, which may be disadvantageous in terms of operation speed.

  When the search server 3 receives the search area range information transmitted in step S202, the search server 3 passes this information to the search processing unit 52. Then, the search processing unit 52 compares the search area range information expressed by the coordinates of the received map image with the map information of the map database 53b by the procedure as the next steps S203 and S204. The coordinate is replaced with position information. That is, the search area range is converted from one represented by the coordinates of the map image to one represented by the position information.

Next, the search processing unit 52 searches for an image file from the map database 53b by using the location information-based search area range data obtained in step S205 according to the procedures of steps S205 and S206, and searches the image file. The file and link information of these image files are captured. Note that steps S205 and S206 are substantially the same as steps S106 and S107 in FIG.
Next, in steps S207 and S208, the search processing unit 52 uses the image file captured in step S205 and its link information to create Web page data for display of the search image list and transmits it to the PC 2. To do. Note that the procedures in steps S207 and S208 only need to be executed by the same processing as steps S108 and S109 in FIG.
The PC 2 receives the search image list display Web page data transmitted in step S208 and passes it to the Web browser. In the Web browser, by processing this Web page data, for example, the search image list display that has been displayed as shown in FIG. 2A is changed to that after the change as shown in FIG. The display is switched to the content corresponding to the size of the search range designation frame MR.

  By the way, as understood from the above description, in order to realize the search service of the present embodiment, it is premised that the search server 3 stores image files to which position information is added. Therefore, an example of a model for the search server 3 to collect image files to which position information is added will be described with reference to FIG.

FIG. 9 shows a video camera 100 and a digital still camera 200 as imaging devices used by an unspecified number of general users. These video camera 100 and digital still camera 200 each have a built-in GPS module. The GPS module here is a component device that can receive radio waves transmitted from GPS satellites, execute positioning calculations, and acquire positioning information.
By incorporating the GPS module in this way, the video camera 100 and the digital still camera 200 receive radio waves from, for example, a plurality of GPS satellites (300A, 300B, 300C), and perform a positioning calculation by a predetermined positioning method. At least current position information can be acquired as positioning information.
Then, in the video camera 100 and the digital still camera 200, the image data obtained by imaging is stored in a storage medium built in itself or a removable storage medium loaded in response to a user performing an imaging recording operation. On the other hand, for example, it is stored in units of files. The image data file (image file) stored in this way has a structure to which, for example, metadata (additional information) having predetermined information items related to the contents of the image data or the situation at the time of shooting is added. In this case, the position information acquired by the GPS module is included in the structure of the metadata.
That is, depending on these imaging devices (video camera 100, digital still camera 200), an image file having the structure shown in FIG. 6 can be obtained.

  Then, the user transmits the image file obtained by photographing and recording with the imaging device by uploading it to the search server 3 via the network. In the search server 3, the image file transmitted in this way is stored in a database by the image file database 53a as described above.

  In addition, the search server 3 can be considered as a model for collecting image files to which position information is added. For example, for an image file that is pasted or linked as a Web page published on the Internet, meta information is reflected in a source such as HTML as well as a file name, for example. Format. In addition, for example, the location of the image file and its position information are acquired by patroling the Internet with a search robot or the like, and the image file database 53a is constructed. In this case, the file entity does not need to exist in the image file database 53a. However, to realize the search processing of the search server 3 according to the above description, for example, many networks known so far This can be done easily by applying search system technology.

As a supplement, a configuration example of the imaging apparatus as the video camera 100 or the digital still camera 200 shown in FIG. 9 will be described with reference to FIG. In this figure, as a configuration of the imaging apparatus, a configuration corresponding to recording of a captured image is extracted and shown. A solid line arrow shown in the drawing indicates a flow of information, such as a video signal, an audio signal, or meta information, which becomes user data to be written and stored in a recording medium (storage medium) in the storage unit. White arrows indicate the flow of information for various controls.
In the imaging apparatus 1 shown in this figure, the imaging unit 11 includes, for example, an optical system (imaging optical system), an imaging device, an image signal processing system before compression coding, and the like. The optical system includes a lens unit including a lens for imaging, a diaphragm, and the like. In addition, the lens unit includes a focus lens for focus adjustment and a zoom lens for zoom (field angle) adjustment. The imaging unit 11 includes a focus lens driving mechanism including a gear, a motor, and the like for moving the focus lens along the optical axis direction for focus adjustment. In addition, a zoom lens driving mechanism including a gear, a motor, and the like for moving the zoom lens along the optical axis direction for zoom adjustment is provided.

In the imaging unit 11, the imaging light obtained by the optical system is imaged on a photoelectric conversion element unit provided in the imaging device and converted into an electrical signal. Examples of the photoelectric conversion element include a CCD (Charge Coupled Device) and a CMOS.
In the imaging unit 11, the electrical signal output from the imaging device is subjected to, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, etc., and converted to a digital signal, and a predetermined image signal by digital signal processing is converted. Apply processing. As this image signal processing, for example, necessary digital signal processing is executed including obtaining a digital video signal corresponding to a captured image from the digital signal.

  Further, in the imaging unit 11, as signal processing, required signal processing according to generation of required control parameters used by the camera control unit 18 for camera control is executed. Here, examples of camera control to be executed by the camera control unit 18 include AF (Auto Focus) control, exposure control, white balance processing, camera shake correction, and the like. As these camera control techniques, known techniques may be employed.

Also, the imaging apparatus 1 of this embodiment, is provided a position detection unit 2 1.
Position detection unit 2 1, for example, a device such as a GPS (Global Positioning System) module, for example, to detect the position at the current time point. The position information detected in this way is represented by, for example, latitude and longitude. In this embodiment, the position information detected by the position detection unit 2 1, are inputted to the camera control unit 18.
In the present embodiment, the position information acquired as described above is used as information to be added to a file of captured image data to be stored in the storage unit 17. That is, as shown in FIG. 6, it is used as position information to be stored in the metadata portion of the image data. Note that the position information may be used for other purposes. As the position detection unit 2 1, be other than comprising a GPS, as long as it employs a configuration can obtain the information of the current position.

The digital video signal finally obtained by the imaging unit 11 is input to the video compression processing unit 12. The video compression processing unit 12 performs compression coding according to a predetermined method on the input video signal. For example, at present, a moving image information compression coding system including MPEG related formats can be adopted for moving images. For still images, various still image compression encoding methods such as JPEG format can be adopted.
The moving image or still image data obtained by the compression encoding by the video compression processing unit 12 is output to the multiplexing processing unit 15.

The sound collection unit 13 includes a predetermined number of microphones and an audio signal processing system that performs predetermined signal processing before compression coding on a sound signal obtained by collecting the sound with the microphones. The In this case, the sound collection unit 13 converts the audio signal from analog to digital format at a predetermined signal processing stage.
The digital audio signal output from the sound collection unit 13 is output to the audio compression processing unit 14.
The audio compression processing unit 14 executes audio compression encoding processing corresponding to the image compression encoding method of the video compression processing unit 12 and outputs the result to the multiplexing processing unit 15. In the case of moving image recording, the video compression processing unit 12 and the audio compression processing unit 14 execute compression encoding processing so as to maintain a so-called lip sync, and each of the compressed encoded video signals ( Compressed video data) and audio signals (compressed audio data) are output.

The camera control unit 18 is a part that collectively represents a predetermined control function for a part that constitutes a camera unit in the imaging unit 11, represented by the AF control, zoom control, and the like described above. Further, information such as a predetermined control result obtained by the camera control unit 18 is taken in by the metadata control unit 20 and used for generating metadata.
The video control unit 23 performs various controls so that the signal processing operation in the video compression processing unit 12 is appropriately executed. For example, data is fetched from the video signal processing unit 12, various detection processes such as motion detection for compression processing, arithmetic processing, and the like are executed. Necessary control for processing is performed. Information such as detection results and control results for the predetermined items obtained in the video control unit 23 can be captured by the metadata control unit 20 and used to generate management information.
In this case, the audio control unit 19 executes various controls for the audio signal processing operation. For this purpose, the audio control unit 19 controls the signal processing operation of the audio compression processing unit 14 based on, for example, predetermined information output according to the signal processing result of the sound collection unit 13 or the like. Information such as a predetermined control result obtained in the audio control unit 19 can also be taken in by the metadata control unit 20 and used to generate metadata.

  The user interface control unit 22 can acquire operation information obtained by a predetermined operation performed on various operators and the like that are provided in the imaging apparatus, and the imaging apparatus can execute an appropriate operation according to the operation. As described above, it is a part for processing operation information. The user interface control unit 22 is configured to transmit operation information obtained in response to a predetermined operation to a required part in each functional part illustrated here.

As illustrated in FIG. 2, the metadata control unit 20 generates information (metadata) as a metadata unit to be attached to the image data. As described above, the metadata control unit 20 generates predetermined control information and detection results output from, for example, the camera control unit 18, the audio control unit 19 , and the video compression processing unit 12 when generating management information. using information such as, other detected position information by the position detection unit 2 1, it collects information items of various metadata including the shooting date and time, to generate. These information items are stored in a predetermined data structure to obtain an information unit as metadata.

  The multiplex processing unit 15 receives compressed encoded data as a moving image or a still image (including compressed audio data in the case of moving image recording) and metadata information. Therefore, the multiplexing processing unit 15 performs, for example, necessary multiplexing processing, composition processing, and the like on the compressed encoded data of the moving image or the still image and the metadata, so that the image as shown in FIG. Image data corresponding to a data structure including a data part and a metadata part is generated. Then, the image data generated in this way is subjected to, for example, recording encoding suitable for the recording format as the storage unit 17 to generate recording data, and the recording data is transferred to the buffer unit 16 to be accumulated. .

The buffer unit 16 transfers a predetermined amount of the accumulated recording data to the storage unit 17 for writing. Thus, the image data shown in FIG. 2, for example, is recorded by sequentially recording data.
The specific configuration of the storage unit 17 includes a predetermined recording medium (storage medium) and a drive capable of recording (and reproducing) data corresponding to the recording medium. In this case, the recording medium may be removable, for example, or may be fixedly mounted together with a drive such as an HDD (hard disk).
As the recording format of the storage unit 17, for example, if the storage unit 17 is compatible with DVD (Digital Versatile Disc), a so-called DVD format such as DVD-Video or DVD-VR (Video Recording) is used. Will be adopted.

  The data interface 24 is a part configured by mounting hardware and software for communicating with an external device by a predetermined data communication method between the imaging apparatus 1 and the external device. The data communication system supported by the communication unit 16 is not particularly limited regardless of whether it is wired or wireless, and the number of corresponding data communication systems should not be limited. At present, as a data communication system, data bus standards such as USB (Universal Serial Bus), IEEE1394, and SCSI (Small Computer System Interface) can be cited as a data communication system. For wireless communication, a short-range wireless communication standard between devices such as Bluetooth (registered trademark) can be cited. In the present embodiment, the image data file stored in the storage unit 17 can be transmitted to the outside via the communication bus by data communication using the data interface 24.

  Note that the configuration shown in this figure is a block diagram showing functions related to recording (and recording) as an imaging device, and is different from the actual hardware configuration. Actually, for example, a part to be imaged by an image sensor, a part to collect sound by a microphone, etc., and a video signal and an audio signal obtained by these parts are converted into digital signals, and predetermined digital signal processing, It consists of a processor that executes encoding processing, multiplexing processing, a drive corresponding to a predetermined recording medium, a CPU, ROM, RAM, and the like, and a microcomputer that executes various required control processing.

Further, the present invention is not limited to the contents as the embodiment described so far, and can be appropriately changed.
For example, in the embodiment, for the sake of simplicity, the information to be searched is a photographic image obtained by being recorded by an imaging device. That is, it is a data file of a still image that has been captured and recorded. However, as can be seen from the fact that the video camera 100 is also shown in FIG. 9, it may be a data file of a moving image recorded and recorded. Even in the case of a moving image file, for example, it is possible to generate a thumbnail image of a still image or a moving image from the data content and create a Web page for displaying a search image list.
Further, the information to be searched for may be an image file other than that obtained by shooting and recording, such as information created by drawing processing or the like. In this case, instead of adding position information at the same time as shooting and recording, an editing operation for adding position information may be performed in the process of creating the image file or after creation. In addition to the image file, other types such as a document file may be used. That is, the search target information as the invention of the present application may be any information that has significant and useful information content for users using the search service, including images and documents. In addition, when position information is added to information to be searched other than such captured images, the position information is treated as indicating the created location or the location according to the information content. It is thought that it is useful.
In the embodiment, a system configuration in which a search is performed by accessing the search server 3 from a PC via a network is taken as an example. For example, in a computer system such as a PC, the image file database 53a and the map If a database equivalent to the database 53b is installed, it is possible to construct a complete image search system within the local computer system.
In the present invention, the temporary search condition input by the user who performs the search by his / her own intention is the character string information. However, the position information is excluded and the information is of a predetermined type and format other than the character string. Is also possible.

DESCRIPTION OF SYMBOLS 1 Imaging device, 2 Personal computer (PC), 3 Search server, 11 Imaging part, 12 Video compression process part, 13 Sound collection part, 14 Audio compression part, 15 Multiplexing process part, 16 Buffer part, 17 Storage part, 18 Camera control unit, 19 audio control unit, 20 management information control unit, 21 position detection unit, 22 user interface control unit, 23 video control unit, 24 data interface, 31 CPU, 32 ROM, 33 RAM, 34 HDD, 34a Web browser 35 Operation input processing unit 36 Display output processing unit 37 Media drive 38 Data interface 39 Network interface 40 Internal bus 41 Operation device 42 Display device 50 Control unit 51 Network interface 52 Content creation Processing section, 53 metadata processing unit, 53 storage unit, 53a an image file database, 53b map database 54 internal bus

Claims (13)

A search method executed by a search device capable of communicating with an information terminal,
Based on the character string information received as the search condition from the information terminal, a search area range determination procedure for determining a search area range,
A map information acquisition procedure for acquiring map information based on the search region range determined by the search region range determination procedure;
An image data acquisition procedure for acquiring image data associated with position information based on the search region range determined by the search region range determination procedure;
A display data generation procedure for generating display data for displaying the map information acquired by the map information acquisition procedure and the image data acquired by the image data acquisition procedure on the display means of the information terminal; ,
A transmission procedure for transmitting the display data to the information terminal,
When the search area range information is received from the information terminal after the display data is transmitted,
The search area range information received from the information terminal is information corresponding to the coordinates about the map image displayed on the information terminal based on the display data, and the information corresponding to the coordinates about the map image. Is converted into position information, and the converted search area range information is used as a search area range used in the search area range determination procedure . The search area range information received from the information terminal is based on an operation for instructing the map image displayed on the information terminal based on the display data to expand or reduce the area range for searching for an image. The search method according to claim 1, which is search area range information. The display data includes information on a reference position corresponding to the character string information,
The search method according to claim 1, wherein the reference position information is displayed on a map image displayed on the information terminal based on the display data. The search method according to claim 1, wherein the search area range determination procedure determines the search area range based on a classification of the character string information received from the information terminal. The search method according to claim 1, wherein the display data includes a thumbnail image of the image data acquired by the image data acquisition procedure. The search method according to claim 1, wherein the display data includes link information to the image data acquired by the image data acquisition procedure. Search area range determining means for determining a search area range based on character string information received as a search condition from an information terminal;
Map information acquisition means for acquiring map information based on the search area range determined by the search area range determination means;
Image data acquisition means for acquiring image data associated with position information based on the search area range determined by the search area range determination means;
Display data generation means for generating display data for displaying the map information acquired by the map information acquisition means and the image data acquired by the image data acquisition means on the display means of the information terminal; ,
Transmission means for transmitting the display data to the information terminal,
When the search area range information is received from the information terminal after the display data is transmitted,
The search area range information received from the information terminal is information corresponding to the coordinates about the map image displayed on the information terminal based on the display data, and the information corresponding to the coordinates about the map image. A search device that converts the search area range information into a search area range that is used by the search area range determination means . A search area range determination procedure for determining a search area range based on character string information received as a search condition from an information terminal;
A map information acquisition procedure for acquiring map information based on the search region range determined by the search region range determination procedure;
An image data acquisition procedure for acquiring image data associated with position information based on the search region range determined by the search region range determination procedure;
A display data generation procedure for generating display data for displaying the map information acquired by the map information acquisition procedure and the image data acquired by the image data acquisition procedure on the display means of the information terminal; ,
A transmission procedure for transmitting the display data to the information terminal;
When the search area range information is received from the information terminal after the display data is transmitted,
The search area range information received from the information terminal is information corresponding to the coordinates about the map image displayed on the information terminal based on the display data, and the information corresponding to the coordinates about the map image. A program that causes a search device to execute a search method that converts a search area range information into a search area range that is used in the search area range determination procedure . A search condition setting method executed by an information terminal capable of communicating with a search device,
The character string information entered as a search condition and transmitting step of transmitting to said search device,
A display data acquired in response to a search area range determined on the basis of the character string information from the search device, a receiving step of receiving the map information and the image data,
A display procedure for displaying the image data together with a map image corresponding to the map information on a display means;
If the search area range corresponding to the map image by a user operation performed on the displayed the map image on said display means is changed, the display means as a search area range information corresponding to the change Search condition setting method information corresponding to the coordinates for the map image to be displayed to execute a transmitting step of transmitting as a new search condition to the search apparatus. The search condition setting method according to claim 9 , wherein the change of the search area range is expansion or reduction of the search area range. The Display data retrieval condition setting method according to claim 9 including the thumbnail image of the image data. Transmission means for transmitting character string information input as a search condition to the search device;
A display data acquired in response to a search area range determined on the basis of the character string information from the search device, receiving means for receiving map information and the image data,
Display control means for displaying the image data together with a map image corresponding to the map information on the display means,
The search area range is variable, and the variable search area range is transmitted to the search apparatus by the transmission means as information corresponding to coordinates about a map image displayed on the display means . A transmission procedure for transmitting character string information input as a search condition to the search device;
A display data acquired in response to a search area range determined on the basis of the character string information from the search device, a receiving step of receiving the map information and the image data,
A display procedure for displaying the image data together with a map image corresponding to the map information on a display means;
If the search area range corresponding to the map image by a user operation performed on the displayed the map image on said display means is changed, the display means as a search area range information corresponding to the change program for executing a transmission step of transmitting information corresponding to the coordinates for the map image as a new search condition to the search apparatus to be displayed on the information terminal device.

Images