JP2005165461A - Information providing device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information providing device which enables a variety of information related to contents of image data from a cellular phone provided to the cellular phone, and an information providing program that makes a computer function as such an information providing device. <P>SOLUTION: A server device 20 has a storage device 20d for storing information related to contents of a number of image data. An information providing process includes step for receiving image data from a cellular phone 10 via a network N, then searching the storage device 20d for the image data most similar to the image data received, and reading the related information matching the detected image data from the storage device 20d. The server device 20 sends the related information read to the cellular phone 10 which has sent the image data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an information providing apparatus for providing various types of information to a user of a mobile phone, and an information providing program for causing a computer to function as such an information providing apparatus.

  As is well known, recent mobile phones have a camera function and an e-mail function. Among these, the camera function is a function for photographing a subject and acquiring the image, and in order to realize this function, the mobile phone includes an objective optical system, an image sensor, an image processor, and a storage medium. Is provided. When a predetermined key on the cellular phone is pressed, the image formed by the objective optical system is converted into image data by the image sensor, and the image data is subjected to various processes by the image processor. The later image data is stored in the storage medium.

  On the other hand, the e-mail function is a function for editing and sending / receiving e-mails, and in order to realize this function, the mobile phone has a program for editing e-mails, instructions for browsing and sending / receiving e-mails. A program for receiving e-mail data, a program for dividing e-mail data into packets, and a program for restoring e-mail data from the packets.

  Note that the electronic mail function described above includes a function for transmitting and receiving an electronic mail with image data attached. Therefore, a user of this type of mobile phone can transmit image data obtained by shooting to another mobile phone or a computer, or receive image data from another mobile phone or a computer.

  As described above, according to a mobile phone having a camera function and an e-mail function, image data can be easily exchanged with other mobile phones and computer users. However, since such exchange of image data is no different from exchanging conventional print photographs, this type of mobile phone can actually be used only as a camera.

  Under such circumstances, some users of mobile phones have requested that information related to the contents of the images can be obtained using images obtained by photographing with the mobile phone.

  The present invention has been made in view of such a demand, and an object of the present invention is to provide various information related to the contents of the image to the mobile phone based on the image data from the mobile phone. And providing an information providing program for causing a computer to function as such an information providing apparatus.

  An information providing apparatus invented to solve the above-described problem is a storage unit that stores related information related to the contents of an image, and receives image data from a mobile phone via a network. A receiving unit, a search unit for detecting image data most similar to the image data received by the receiving unit from the storage unit, and related information corresponding to the image data detected by the search unit from the storage unit It is characterized by comprising a reading unit for reading and a transmission unit for transmitting the related information read by the reading unit to a mobile phone that is a transmission source of the image data received by the receiving unit.

  With this configuration, when image data is received from a mobile phone, related information related to the content of the image indicated by the image data is transmitted to the mobile phone. Therefore, the user of the mobile phone can acquire related information related to the content of the image indicated by the image data.

  However, if the mobile phone has a camera function for acquiring image data, the user can take a picture of the object when he / she wants to know a lot about the object. By simply transmitting image data of an image of the object to the information providing apparatus, information related to the object can be easily obtained.

  In the information providing apparatus according to the present invention, the related information can be a URL [Uniform Resource Locator] of a web page having content related to the content of the image indicated by the image data. In this case, since the URL is transmitted to the mobile phone from which the image data is transmitted, the user of the mobile phone can easily access the web page of the URL, and as a result, the image It is possible to easily obtain related information related to the content of the image indicated by the data.

  In addition, the image data and URL stored in the storage unit can be used as they are stored in a server device operated by an ISP [Internet Service Provider] operator or a search engine operator. . Accordingly, such ISP providers and search engine operators simply provide a service that provides information related to the contents of the image indicated by the image data to the user of the mobile phone having the image data transmission function. be able to.

  By the way, when transmitting a URL to a mobile phone that is the source of image data, an e-mail in which the URL is described may be sent to the mobile phone, but HTML [HyperText Mark-up with the URL embedded in a tag] Language] Mail should be sent. In this way, the user of the mobile phone that has received the HTML mail can easily access the web page simply by clicking on the clickable button in the electronic mail.

  In addition, an information providing program invented to solve the above problems includes a storage unit and a network for storing a computer in a storage device in association with a large number of image data and related information related to the contents of the image. Receiving means for receiving image data from any mobile phone through a communication device connected to the mobile phone, search means for detecting image data most similar to the image data received by the receiving means from within the storage device, Reading means for reading related information corresponding to the image data detected by the searching means from the storage device, and communicating the related information read by the reading means to the mobile phone that is the transmission source of the image data received by the receiving means It is characterized by functioning as transmission means for transmitting through the apparatus.

  Therefore, according to this information providing program, the computer can function as an apparatus that functions in the same manner as the information providing apparatus of the present invention described above.

  As described above, according to the present invention, various types of information related to the contents of the image can be provided to the mobile phone based on the image data from the mobile phone.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a schematic diagram of a network system to which the present invention is applied. This network system includes one or more mobile phones 10 and a server device 20, and these mobile phones 10 and the server device 20 are connected to each other via a network N.

  The mobile phone 10 is a mobile terminal device having a function for performing wireless communication with a relay device configuring the network N. FIG. 2 is an internal configuration diagram of the mobile phone 10.

  The cellular phone 10 includes a CPU 10a, SRAM 10b, a display device 10c, a key input device 10d, a ringing tone generating device 10e, a ringing tone speaker 10f, a calling speaker 10g, a microphone 10h, an external interface (IF) device 10i, and a wireless communication device. 10j, an area image sensor 10k, an image processing device 101, a memory card drive 10m, and a flash memory 10n. The hardware 10a to 10n are connected to each other via a bus B.

  The CPU 10a is a central processing unit for controlling the entire mobile phone 10. The SRAM 10b is a main storage device in which a work area is expanded when the CPU 10a executes various programs.

  The display device 10c is a device for displaying various screens based on the screen data generated by the CPU 10a. Specifically, the display device 10c is a liquid crystal display (LCD) or an LCD driver chip. The key input device 10d is a device for receiving an input from an operator, and specifically, is a numeric keypad for numeric input and character input, a special key, a cross key, and a key scan circuit.

  The ring tone generator 10e is a synthesizer that generates an electrical signal based on audio data. The ringing sound speaker 10f is a device that converts an electric signal from the ringing tone generating device 10f into a sound wave.

  The call speaker 10g is an output device that converts sound data into sound waves and outputs the sound. The microphone 10h is an input device that converts sound waves to be input into sound data. The external IF device 10i is an interface device for detachably mounting a battery charging or data input / output cable connector.

  The wireless communication device 10j is a device for wirelessly communicating with the nearest relay device among the relay devices described above, and more specifically, an antenna unit for transmitting and receiving a carrier wave signal, and a baseband signal of the carrier wave signal A radio frequency (RF) unit that modulates the signal and demodulates the baseband signal from the carrier signal, and a baseband unit that converts the baseband signal and digital data (voice data, packets, and control data) Has been.

  The area image sensor 10k is a single-plate image pickup device having an image pickup surface constituted by a number of pixels arranged two-dimensionally. A color filter is on-chip on the imaging surface, and an objective optical system is set near the imaging surface.

  The image processing device 101 is a device for processing image data output from the area image sensor 10k, and specifically, performs processing such as format conversion, image quality adjustment, and compression on the image data.

  The memory card drive 10m is a device for writing image data to a memory card that is a storage medium and erasing image data in the memory card. The memory card drive 10m includes a mechanism for detachably mounting a memory card.

  The flash memory 10n is a storage device that stores various data and various programs. The program stored in the flash memory 10n includes an operating system (OS) program for integrated management of hardware and software, and also includes a mailer 11 and a web browser 12. Yes.

  The mailer 11 is a program for realizing on the mobile phone 10 a function for editing an e-mail and a function for sending and receiving e-mail. Further, the mailer 11 has a function of transmitting an e-mail attached with image data.

  Note that a mailer 11 that complies with a certain communication protocol defined by a certain wireless telephone company (first-class telecommunications carrier) can directly send an e-mail attached with image data to another mobile phone or a computer. Can not. However, in this communication protocol, the image data attached to the e-mail is once stored in a server device dedicated to image data constructed by the wireless telephone company, and the server device attaches the image data to the e-mail. And send the e-mail to its original destination. Accordingly, the mailer 11 can substantially transmit an electronic mail with image data attached to another mobile phone or computer.

  The web browser 12 is a program for acquiring web page data, image data, and the like from a website of URL [Uniform Resource Locator] designated by the operator and displaying the web page, the image, and the like on the display device 10c.

  The server device 20 is a computer used by ISP (Internet Service Provider) operators and search engine operators to provide services. FIG. 3 is an internal configuration diagram of the server device 20.

  The server device 20 includes a CPU 20a, a RAM 20b, a communication control device 20c, and a storage device 20d. These hardware 20a to 20d are connected to each other via a bus B.

  The CPU 20a is a central processing unit for controlling the entire server device 20. The RAM 20b is a main storage device in which a work area is expanded when the CPU 20a executes various programs. The communication control device 20c is a device for transmitting / receiving data to / from a router on the network N, and specifically, a network adapter such as a network interface card (NIC) or an Ethernet adapter.

  The storage device 20d is a device that stores various data and various programs. The data stored in the storage device 20d includes image data 21, an image management table 22, a digest database (DB) 23, and a URL ranking table 24. In addition to the OS program, the program stored in the storage device 20d includes a mailer 25, an OCR [Optical Character Reader] program 26, an information providing program 27, and a count program 28.

  Hereinafter, each data 21-24 and each program 25-28 memorize | stored in this memory | storage device 20d are demonstrated in order.

  The image data 21 is image data (in the web page) attached to the web page data (created by the ISP company and each user) stored in the web server device operated by the ISP company. A copy of image data of an image to be pasted), or a copy of image data attached to web page data cached for a search engine provided by an ISP. Further, the image data 21 includes image data attached to an e-mail sent from the mobile phone 10 to the server device 20 as will be described later. The storage device 20d that stores the image data 21 corresponds to a storage unit, and the server device 20 that stores the image data 21 in the storage device 20d corresponds to a storage unit.

  The image management table 22 is a table for managing each image data 21 stored in the storage device 20d. FIG. 4 is a diagram showing an example of the data structure of the image management table 22. The image management table 22 in FIG. 4 has the same number of records as the image data 21 stored in the storage device 20d. Each record has fields of “file name”, “address”, “alt attribute”, and “keyword”.

  In the “file name”, a path name and a file name of each image data 21 in the server device 20 are recorded. The “address” is the URL of the website that originally possesses the web page data accompanied by the image data 21 or the e-mail address of the mobile phone 10 that is the transmission source of the image data 21.

  In the “alt attribute”, the alt attribute value in the <img> tag that specified the image data 21 in the web page data is recorded. However, when the image data 21 is received from the mobile phone 10, the alt attribute value itself does not exist, so the “alt attribute” of the image data 21 is blank. In the “keyword”, a keyword corresponding to the content of the image indicated by the image data 21 is recorded.

  The digest DB 23 is a database for managing the URL that is the original location of the web page data and information related to the content. The digest DB 23 may use a database for managing the web page data constructed by the ISP company or each user in the web server device operated by the ISP company. A database for managing web page data cached for the search engine may be used.

  FIG. 5 is a diagram showing an example of the data structure of the digest DB 23. As shown in FIG. The digest DB 23 of FIG. 5 has the same number of records as the web page data managed in the database in the ISP server of the ISP provider, or the same number of records as the web page data cached for the search engine. Have. Each record has at least fields of “URL”, “site name”, “details”, and “degree of conformance”.

  In “URL”, a URL that is the original location of the web page data is recorded. In “site name”, a site name to be described in the link button when a link is established with each URL as a link destination is recorded. In “Details”, text extracted from the contents of the web page data at each URL is recorded. In “Adaptability”, the adaptability, which is a ratio occupied by the number of jumps to the URL out of the total number of jumps to each URL from an e-mail described later within a predetermined period, is recorded.

  The URL ranking table 24 is a table for recording the number of jumps to each URL within a predetermined period. FIG. 6 is a diagram showing an example of the data structure of the URL ranking table 24. The URL ranking table 24 of FIG. 6 has the same number of records as all URLs recorded in the digest DB 23. Each record has fields of “URL” and “count number”.

  Each URL is recorded in “URL”. In the “count number”, a count number that is incremented by 1 when the URL is linked to an e-mail described later as a link destination and the URL is selected as a jump destination from the e-mail is recorded. The

  The mailer 25 is a program for realizing on the server device 20 a function for sending and receiving electronic mail. In addition, the mailer 11 has a function of receiving an e-mail attached with image data.

  The OCR program 26 is a program for realizing on the server device 20 a function of detecting a character image from an image indicated by image data and acquiring text data of a character corresponding to the detected character image.

  The information providing program 27 is a program for providing information related to the image data based on the image data received from the mobile phone 10. Details of the processing of the information providing program 27 will be described later.

  The count program 28 is a program for counting up the value of the “count number” field in the URL ranking table 24 and updating the value of the “fitness” field 23 of the digest DB. The processing contents of the count program 28 will also be described later.

  The contents of processing executed in the network system configured as described above will be described below.

  The CPU 20a of the server device 20 receives an e-mail from the mobile phone 10 through the communication control device 20c and the mailer 25 (the CPU 20a executing the mailer 25 corresponds to a receiving means. At this time, the CPU 20a and the communication control device 20c are The information providing program 28 is read from the storage device 20d into the RAM 20b using the reception as a trigger. Then, an information providing process for processing the electronic mail is generated in the server device 20. Note that e-mails may be simultaneously received from a plurality of mobile phones 10, but in such a case, the server device 20 generates one information providing process for each e-mail. As a result, a large number of information provision processing processes are executed in parallel.

  FIG. 7 is a flowchart showing the contents of the information providing process. As shown in FIG. 7, the information provision processing process includes steps S101 to S111.

  In the first step S101, the CPU 20a determines whether or not the email is from the mobile phone 10 based on the sender email address in the header of the email to be processed. Note that a domain name indicating a wireless telephone company that uses the mobile phone 10 as a wireless telephone terminal device is used in the e-mail set in the mobile phone 10. Therefore, it is possible to determine whether or not the e-mail is from the mobile phone 10 based on the domain name. When the processing target e-mail is not from the mobile phone 10 (S101; NO), the CPU 20a ends the information provision processing process, and the processing target e-mail is from the mobile phone 10 ( (S101; YES), the process proceeds to step S102.

  In step S102, the CPU 20a determines whether image data is attached to the processing target email. If the image data is not attached to the processing target email (S102; NO), the CPU 20a ends the information provision processing process, and if the image data is attached to the processing target email (S102; YES). ), The process proceeds to step S103.

  In step S103, the CPU 20a activates the OCR program 26 and executes the OCR process with the image data attached to the processing target email as the processing target. After executing this OCR process, the CPU 20a advances the process to step S104.

  In step S104, the CPU 20a determines whether text is detected by the OCR process. When the text is detected by the OCR process (S104; YES), the CPU 20a advances the process to step S107. When the text is not detected by the OCR process (S104; NO), the CPU 20a advances the process to step S105.

  In step S105, the CPU 20a detects, from the storage device 20d, the image data 21 that is most similar to the image data that has been processed by the OCR process. Note that such a technique related to detection of similar image data is generally known as described in, for example, [http://www.ricoh.co.jp/tech/23/index.html]. Therefore, description of the technique is omitted here. In addition, CPU20a which performs this step S105 is corresponded to a search part and a search means.

  In the next step S106, the CPU 20a identifies a record corresponding to the image data 21 detected in step S105 from the records in the image management table 22, and stores the “alt attribute” and “keyword” fields in the record. Read the text of. The CPU 20a executing step S106 corresponds to a reading unit and reading means. After reading, the CPU 20a advances the process to step S107.

  In step S107, the CPU 20a searches the digest DB 23 using the text detected in step S103 or the text read in step S106 as a search condition. In this search process, the CPU 20a determines whether or not a search condition exists in the “detail” field for each record in the digest DB 23, and the search condition exists in the “detail” field. In addition, information in the “site name”, “URL”, and “fitness” fields of the record is read and stored as one record in a work table prepared in advance in the RAM 20b.

  In the next step S108, the CPU 20a rearranges the records recorded as search results in the work table in the RAM 20b in order from the largest value in the “fitness” field.

  In the next step S109, the CPU 20a creates an e-mail based on each record recorded in the work table in the RAM 20b. More specifically, the CPU 20a embeds at least one tag corresponding to each record in the work table in the body of the e-mail in the same order as in the work table. This tag is a tag for displaying the site name as a clickable link button which is an input means when jumping to the URL. However, when this link button is clicked, the web browser 12 according to this tag does not jump directly to the URL, but notifies the server device 20 of the URL once and receives the notification. Jumps to the URL in response to a redirect instruction from. After generating the body of the e-mail by embedding such a tag, the CPU 20a generates a header with the transmission source e-mail address of the processing target e-mail as the transmission destination, and adds the header to the body. Generate email.

  In the next step S110, the CPU 20a transmits the electronic mail generated in step S109 through the communication control device 20c and the mailer 25. In addition, CPU20a which performs step S109 and S110 is corresponded to a transmission part and a transmission means.

  In the next step S111, the CPU 20a stores the image data targeted for the OCR process in step S103 in the storage device 20d. In addition, the CPU 20a adds a record corresponding to the image data to the image management table 22. The record added to the image management table 22 has the text detected in step S103 or the text read in step S106 in the “keyword” field, and the transmission destination of the email transmitted in step S110. The e-mail address is stored in the “address” field, and the path name and file name of the image data are stored in the “file name” field. After execution of step S111, the CPU 20a ends the information provision process.

  Since such information providing processing is performed in the server device 20, the network system of the present embodiment operates as described below.

  When the user of the mobile phone 10 wants to obtain various kinds of information about an existing object, he / she takes an image of the object by operating the key input device 10d, and transfers the image data of the object to the mobile phone 10 Create it. For example, to obtain information about a dachshund that is in front of the user, the dachshund is photographed. FIG. 8 is a diagram showing an example of this dachshund image. Then, the user operates the key input device 10d to transmit an e-mail attached with the image data to the server device 20.

  Then, the server device 20 performs OCR processing on the image data (S103), and can detect the text from the image indicated by the image data and can detect the text from the image. If not, the text corresponding to the image data 21 similar to the image data is acquired (S104 to S106). For example, since there is no text in the image shown in FIG. 8, the image data 21 most similar to the image data of this image is detected. For example, if it is determined that the image data of the file name 000031.jpg shown in the image management table 22 of FIG. 4 is most similar to the image data of the image of FIG. 8, the alt attribute value of 000031.jpg The keyword is acquired by the server device 20.

  Then, the server device 20 searches the digest DB 23 using the acquired text as a search condition (S107), sorts the detected records in order of suitability (S108), and then lists the search results in the order after sorting. The e-mail is transmitted to the mobile phone 10 that has transmitted the e-mail with the image data attached (S109, S110). For example, when the record of the site name “Pet de Nippi” shown in the digest DB 23 of FIG. Are listed. FIG. 9 is a diagram showing an example of the contents of the e-mail 31. In the e-mail of FIG. 9, several site names 31a including the site name “pet de nippi” are listed. On the e-mail 31, each site name 31a is shown as a clickable link button. Then, when the user selects and clicks one of the site names 31a indicated as the link button in the e-mail in FIG. 9, the web browser 12 acquires the web page data of the site and obtains the web page. Is displayed on the display device 10c, the user can easily obtain information about the dachshund shown in FIG.

  By the way, when the server device 20 is operated, the server device 20 is in a state where a count process is generated. The count process is generated by the count program 28 being read from the storage device 20d into the RAM 20b by the CPU 20a.

  When the user of the mobile phone 10 clicks the site name indicated as the link button in the email received from the server device 20, the URL of the site is notified to the server device 20, and the count process is performed together with the notification. The process is kicked. In the kicked count process, the count number corresponding to the URL in the URL ranking table 24 in the storage device 20d is incremented by 1, and the mobile phone 10 that has notified the URL is instructed to redirect to the URL.

  The count processing process continues such increment processing for a predetermined period such as two weeks. Then, after the elapse of a predetermined period, the count processing process adds all the count numbers in the URL ranking table 24 to obtain the total count number, and divides all the URL count numbers by the total count number. As a result, a new fitness level for each URL is calculated. Then, the count process updates the “fitness” field corresponding to each URL in the digest DB 23 with the calculated new suitability. After the update, the count processing process once disappears, and a new count process is generated by the server device 20. The newly generated count processing process clears all the count numbers in the URL ranking table 24 and starts the above-described increment processing.

  By performing such a counting process in the server device 20, the fitness of the digest DB 23 is updated. Therefore, even when the user of the mobile phone 10 transmits to the server device 20 to obtain various information using the same image data, the order of the site names indicated in the e-mail received from the server device 20 is the same. , Will be replaced.

Schematic diagram of a network system to which the present invention is applied Mobile phone internal configuration diagram Server device internal configuration diagram The figure which shows an example of the data structure of an image management table The figure which shows an example of the data structure of digest DB The figure which shows an example of the data structure of a URL ranking table Flow chart showing the contents of information provision processing Diagram showing an example of a dachshund image Figure showing an example of the contents of an email

Explanation of symbols

10 mobile phone 10a CPU
10b SRAM
10c Display device 10d Key input device 10e Ring sound generation device 10j Wireless communication device 10k Area image sensor 10l Image processing device 10m Memory card drive 10n Flash memory 11 Mailer 12 Web browser 20 Server device 20a CPU
20b RAM
20c communication control device 20d storage device 21 image data 22 image management table 23 digest DB
24 URL ranking table 25 Mailer 26 OCR program 27 Information providing program 28 Count program

Claims (5)

For each of a large number of image data, a storage unit for storing related information related to the contents of the image,
A receiver for receiving image data from a mobile phone via a network;
A search unit for detecting image data most similar to the image data received by the receiving unit from the storage unit;
A reading unit for reading out related information corresponding to the image data detected by the search unit from the storage unit; and
An information providing apparatus comprising: a transmission unit that transmits related information read by the reading unit to a mobile phone that is a transmission source of image data received by the reception unit. The storage unit stores, for each of a large number of image data, a keyword for the content of the image and related information related to the content of the image,
The search unit determines the presence or absence of text in the image indicated by the image data received by the receiving unit, and if there is text in the image, stores the same keyword as the text The information providing apparatus according to claim 1, wherein the information providing apparatus detects the information from a unit. The storage unit stores, for each of a large number of image data, related information related to the content of the image and frequency information indicating the frequency of use of the image data,
The transmitting unit rearranges the related information read by the reading unit in order from the most frequently used one corresponding to the related information to the lowest one, and the related information is listed in the order after the rearrangement. The information providing apparatus according to claim 1, wherein the mail is transmitted to a mobile phone that is a transmission source of the image data received by the receiving unit. An acquisition unit that acquires information indicating which related information is selected in the mobile phone among the related information transmitted by the transmission unit to the mobile phone; and
The information providing apparatus according to claim 3, further comprising an update unit that updates frequency information for each of the image data in the storage unit based on the information acquired by the acquisition unit. Computer
Storage means for storing a large number of image data and related information related to the contents of the image in association with each other in a storage device;
Receiving means for receiving image data from any mobile phone through a communication device connected to the mobile phone via a network;
Search means for detecting image data most similar to the image data received by the receiving means from within the storage device;
Reading means for reading out related information corresponding to the image data detected by the search means from the storage device; and
An information providing program for causing the related information read by the reading means to function as a transmitting means for transmitting to a mobile phone that is a transmission source of image data received by the receiving means through a communication device.

Images