JPH10260916A - Image data processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the user interface for the use of an electronic mail etc. SOLUTION: The user of a portable terminal 1 makes notes of a document to be sent as an E-mail, the mail address of a terminal 9, and information indicating that service requested of a host device 4 is 'mail transmission' on a paper surface, etc., and photographs the paper surface, etc., by using an image input part (camera) 2. The portable terminal 1 transfers the captured image data to a host device 4. The host device 4 analyses the received image data by using an image recognition part 5 and actuates a mail transmission and reception part 6 once recognizing that the service that the portable terminal 1 is requesting is 'mail transmission'. The main transmission and reception part 6 sends the memory contents included in the image data to the terminal 9 specified with the mail address included in the image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a system for processing image data input using a certain terminal device by another information processing device.

[0002]

2. Description of the Related Art In recent years, electronic mail (E-mail) has been rapidly spreading for business use or personal use. E-mail is used in a wide variety of forms, for example, from a small in-house mail system that transfers messages and the like within a LAN (local area network) to a system that sends e-mail to desired parties around the world via the Internet. is there.

[0003] When sending an e-mail, the user normally creates a letter by keying in a document to be sent and a destination address, and then clicks a "send" button on the e-mail application, for example. Send the letter to the network. Alternatively, instead of keying in a document to be transmitted, text printed on paper or the like should be read as image data using an image scanner or the like, and transmitted by executing character recognition processing on the image data. Sometimes you get a document. Upon receiving the letter (actually, a packet containing the letter), the network transfers the letter to the destination specified by the user according to the destination address set in the letter.

As described above, conventionally, when an electronic mail is transmitted, a character string entered by a user through a key is transmitted to the network, or at least information converted into a character code string is transmitted to the network. . Further, the user inputs a key to the transmission destination address.

[0005]

However, there have been some voices calling for a simpler user interface when using electronic mail. For example, many users want to input a character string by a method other than key input. As a character input method other than key input, a form in which handwriting input is performed on a display screen of a terminal device using a dedicated pen is known.However, it may be difficult to draw a fine pattern. If you want to send the contents as an e-mail as it is, you need to enter it again with a pen, which is troublesome. In many cases, a user sends an e-mail from a destination using a mobile terminal, but even if a character string written on paper or the like is captured as image data by any means, the mobile terminal is If they do not have character recognition software, they will not be able to send the captured content as a mail document. In general,
High-performance character recognition software has a large program scale, and it is not desirable to install such a program in a simple portable terminal or the like.

An object of the present invention is to improve a user interface when using electronic mail or the like.

[0007]

SUMMARY OF THE INVENTION The present invention relates to a system for processing image data input using a certain terminal device by another information processing device, comprising an image as transmission information and a destination of the transmission information. A mobile terminal that captures image data including characters that specify characters, receives image data captured by the mobile terminal, recognizes a character included in the image data, and includes a character included in the received image data. An information processing apparatus for transmitting transmission information to a destination apparatus designated by the recognized character.

According to the above arrangement, the contents (transmission information such as text) to be transmitted to the desired terminal and the address of the desired terminal (characters for designating the transmission destination) are written on paper or the like, and the memo is written. If the information is transferred to the information processing device, the information processing device performs character recognition processing on the received image data to detect the transmission destination, and the information captured by the portable terminal to the transmission destination (Contents written on a memo, etc.).

The service system for transferring information to the desired terminal is, for example, electronic mail. The information designating the service method is included in the image data transferred from the mobile terminal to the information processing device.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a system configuration diagram of the present embodiment. The mobile terminal 1 has a communication function, and can request another device connected to the network 3 for data processing, receive the result, and display the result. The portable terminal 1 transmits and receives data wirelessly via a nearest base station, transmits and receives data to and from a device (optical link unit) having a communication function by Ir (infrared) communication, or uses a wired connection. Is connected to the network 3 by a method of transmitting and receiving data. In addition, the mobile terminal 1
Has an image input unit 2 composed of, for example, an electronic camera, and has a function of sending image data captured using the image input unit 2 to another device (for example, a host device 4) connected to the network 3 to request image processing. have.

Network 3 is a public telephone network, PHS
It is a network or a LAN, and accommodates the host device 4. The host device 4 is a server machine, and the mobile terminal 1
Executes data processing according to the request transferred from. The host device 4 includes an image recognition unit 5, a mail transmission / reception unit 6, and a facsimile transmission / reception unit 7. The image recognition unit 5 performs a pattern recognition process or a character recognition process on the image data sent from the mobile terminal 1, and detects a predetermined pattern and a character string included in the received image data. The mail transmitting / receiving unit 6 has, for example, a function of transmitting / receiving an electronic mail to / from another terminal 9 connected via the Internet 8, and can transmit a character string recognized by the image recognition unit 6 to the terminal 9. . The FAX transmitting / receiving unit 7 has, for example, a function of transmitting / receiving a FAX to / from the FAX terminal 10 connected via the network 3, and can transfer the image data sent from the portable terminal 1 to the FAX terminal 10. it can.

In the above configuration, when the user of the portable terminal 1 wants to send the contents of a memo written on paper or the like to the terminal 9 as an e-mail, the mail address of the terminal 9 and the host device 4 are added to the memo. Information for identifying that the requested service is “mail transmission” is included as image information on the paper, and the paper is captured using the image input unit 2 (camera attached to the mobile terminal 1), and the captured data is captured. The image data is transferred to the host device 4. The host device 4 analyzes the received image data using the image recognition unit 5 and, when recognizing that the service requested by the mobile terminal 1 is “mail transmission”, activates the mail transmission / reception unit 6 and activates the mail transmission / reception unit 6. The contents of the memo included in the image data are transmitted to the mail address included in the image data.

Similarly, the user of the portable terminal 1 uses the contents of the memo written on a sheet of paper as a facsimile,
When the user wants to send a fax to the memo, the telephone number of the FAX terminal 10 and information for identifying that the service requested by the host device 4 is “FAX transmission” are included as image information on the paper. The paper is used as the image input unit 2
And transfers the captured image data to the host device 4. When recognizing that the service requested by the mobile terminal 1 is “FAX transmission”, the host device 4 activates the FAX transmission / reception unit 7 and replaces the memo included in the image data with the image data. Fax transmission to the included telephone number.

The host device 4 may be constituted by one computer, or may be constituted by an image processing server device, a mail server device, a facsimile server device and the like, each of which is an independent computer.

As described above, in the system of this embodiment,
If you want to send e-mail or fax,
It is only necessary to photograph the recording surface containing the content to be transmitted with a camera or the like and transmit it as image data to the host device 4. Therefore, the user can receive an e-mail transmission service or a facsimile transmission service simply by writing down desired characters or figures on paper or the like and photographing the memos with a camera. Further, the mobile terminal 1 only needs to have a function of capturing image data and a communication function as configuration requirements for receiving the service as described above, so that weight and cost can be reduced.

FIG. 2 is an external view of the portable terminal 1. The mobile terminal 1 includes an LCD display unit 11, a camera 12, an IC card slot 13, and a wireless communication antenna 14. Further, in order to support other communication methods, a socket for Ir communication, a socket for wired communication, and the like are provided. The LCD display unit 11 has a touch panel configuration, and the mobile terminal 1 can process handwritten input information.

FIG. 3 is a configuration diagram of the portable terminal 1. CP
U21 executes a program stored in the storage device 22 (ROM and RAM). The CPU 21 and the storage device 22 are connected to each other via a bus 23.

The storage device 22 is composed of a semiconductor memory, a magnetic recording medium, or an optical recording medium, and stores programs and data. The storage device 22 may be fixedly provided in the mobile terminal 1 or may be detachably mounted.

The recording medium driver 24 is connected to the bus 23, and reads data stored in a portable recording medium (including a semiconductor memory, a magnetic disk, an optical disk, and a magneto-optical disk) 25, or reads out data stored in a portable recording medium. This is a device for writing data to the recording medium 25. As an example of the portable recording medium 25, an IC card is assumed. CPU
The program 21 can also execute a program stored in the portable recording medium 25.

The program and data to be recorded in the storage device 22 may be configured to be received from another device connected via a communication line or the like and recorded. The programs and data stored in the provided storage device may be used via a communication line or the like.

The unit corresponding to the LCD display unit 11 is:
Liquid crystal display (LCD) 31, Liquid crystal display 3
1, an LCD driver 3 for outputting information stored in the memory 32 to the liquid crystal display 31 under the control of the LCD controller 34.
3. L controlling the memory 32 and the LCD driver 33
It comprises a CD control unit 34, a touch panel unit 35, and a touch panel control unit 36 that notifies the CPU 21 of input information detected by the touch panel unit 35.

The camera 12 is, for example, a CCD camera, whose output is converted into bitmap digital data by an A / D converter 37 and written into a memory 38. The camera control unit 39 outputs the image data stored in the memory 38 to the bus 23 according to an instruction from the CPU 21.

When transmitting data from the portable terminal 1, the communication control unit 40 generates a transmission packet in accordance with an instruction from the CPU 21 and passes it to the wireless transceivers 41 and 42 or the wired transceiver 43. When data is received, data stored in a packet received via the wireless transceiver 41 or 42 or the wired transceiver 43 is output onto the bus 24. Wireless transceiver 41
Are connected to the wireless communication antenna 14 shown in FIG. 3 and exchange wireless data with the wireless base station 4. The wireless transceiver 42 is a transceiver for performing Ir communication, and the wired transceiver 43 is, for example, a modem. Wireless transceiver 42 and wired transceiver 4
3 is optionally attached and detached. In addition, the mobile terminal 1
Further, a clock 44 is provided.

FIG. 4 is a flowchart of a process for requesting a desired service by transferring image data captured by a camera in the portable terminal 1 to the host device 4.
Programs for realizing the functions shown in this flowchart are stored in the storage device 22 in the form of program codes readable by the CPU 21.

The following flowchart shows the processing after the portable terminal 1 detects some input in the input waiting state. In this embodiment, an example will be described in which characters or the like written by a user are photographed by a camera.

In step S1, it is checked whether or not the detected input is an input from the camera 12. If the input is from the camera 12, the processing from step S2 is executed. Perform other processing corresponding to the input. As described above, when the user shoots using the camera 12 of the mobile terminal 1, the processing after step S2 is started. When there is a camera input,
Time information output from the clock 44 is held.

In step S2, the image data captured by the camera 12 is displayed on the LCD display unit 11.
That is, the image data captured by the camera 12 is written into the memory 32 of the LCD display unit 11, and the image data is displayed on the liquid crystal display 31. This allows the user to view the captured image on the spot. FIG. 5A is a display example of image data captured by a user through camera shooting. Here, the user is paper or the like, as e-mail statement, "Thank you Hello .E-mail. Although next week of the schedule, ..."
In this example, the user writes "Email" as a service requested to the host apparatus 4 and "abc@xyz.co.jp" as a mail destination address. The designated service and the address are respectively enclosed in frame-type parentheses.

As described with reference to FIG. 1, in order to transfer image data to the host device 4 and cause the host device 4 to perform data transmission processing in accordance with information included in the image data, The information specifying the service and the information specifying the destination of the mail must be identifiable for the mail text (mail content). In the example of FIG. 5 (a), rules for enclosing the information for specifying the service and the information for specifying the destination of the mail with frame-type brackets are determined. Alternatively, a rule may be provided to underline information specifying a service and information specifying a mail transmission destination. Further, a method of making the service type correspond to the color may be used. For example, the mail transmission service is determined to be “red”, the fax transmission service is determined to be “blue”, and rules are set such that the contents written in those colors specify the transmission destination. In the example shown in FIG. 5B, since “abc@xyz.co.jp” is written in red, the host device 4 analyzes the image data including this image to request the user. It can be recognized that the service is “e-mail transmission” and the destination address is “abc@xyz.co.jp”.

An image taken by the camera 12 is converted to a desired facsimile
To send a fax to the terminal, as shown in FIG. 6 (a), write "FAX send" as a service requested to the host device 4 near a memo (here, a map) to be sent, and write the character string. Enclose in frame type parentheses. In addition, the telephone number of the destination FAX terminal is also written together with the frame type parentheses.

When the content to be transmitted as an e-mail or a facsimile is photographed by the camera 12, a predetermined format may be used. The example shown in FIG.
This is a format for entering the content to be transmitted as an e-mail, and has an address entry area and an e-mail text entry area.

The services requested by the host device 4 include those that transmit data to a desired device such as mail transmission or facsimile transmission, and those that arrive at a mail server (mail transmission / reception unit 6 in FIG. 1). It is also possible to request that a mail addressed to oneself be downloaded to the portable terminal 1. In this case, as shown in FIG. 7, "mail inquiry" is written as a service requested to the host device 4 on a sheet of paper or the like, and the character string is surrounded by frame-type parentheses. To read only mail from a specific person, write "from K" or the like on the paper. Then, the memo is written on the camera 12
, And transmits the captured image data to the host device 4.

Returning to the flowchart of FIG. Step S
At 3, it is determined whether or not the image data input by the camera has been corrected. Here, an example of the operation of the user when the screen shown in FIG. 8A is displayed by the processing of step S2 will be described. When the user wants to modify the content of the display screen, the user inputs the content of the modification on the LCD display unit 11 by handwriting using a handwriting pen. For example, in the sentence displayed on the LCD display section 11, "CASIO's portable terminal released this month has enhanced mail functions and ... by the way ..." Is unnecessary, a deletion symbol is written to "By the way ..." on the LCD display unit 11 using a handwriting pen as shown in FIG. 8 (b). When the correction of the display content is completed, the user inputs that the correction has been completed. This processing is performed, for example, by displaying an image captured by the camera 12 on the LCD display unit 11.
Is displayed together with the "correction end" button, and the user presses the button with a handwriting pen. If there is no correction, camera 1
The user presses the modification end button without modifying the image captured in step 2.

Steps S3 and S4 are executed in accordance with the operation of the user. That is, when it is detected in step S3 that the correction end button has been pressed, the process proceeds to step S4. If the user presses the correction end button without correcting the image data displayed on the LCD display unit 11, step S4 is skipped and the process proceeds to step S5.

In step S4, the image data is updated according to the pen input from the user. That is, the locus pattern pressed on the LCD display unit 11 by the pen input is overwritten in the memory 32 as bitmap data. At this time, since the image data currently displayed on the LCD display unit 11 is stored in the memory 32, the image data captured by camera shooting and the trajectory pattern written by pen input are combined.

In step S5, a screen for allowing the user to specify whether or not to transmit the image captured by the camera to the host device 4 is displayed on the LCD display unit 11. At this time, the image data stored in the memory 32 is saved in the RAM of the storage device 22. Then, the image data of the screen for prompting the user to specify the desired service is read out from the ROM of the storage device 22 and written in the memory 32. The user instruction input screen includes, for example, buttons of “send image”, “save”, and “cancel”.

When the user inputs an instruction on the user instruction input screen, that is, when the user presses one of the displayed buttons, the processing from step S6 is executed. First, in step S6, an instruction from the user is recognized. This processing detects which button the user has pressed. In step S7, it is determined whether the user's instruction is mail transmission or fax transmission. Here, when the image transmission button is pressed, it is determined that the user's instruction is mail transmission or fax transmission, and the process proceeds to step S8.

In step S8, when transferring the image data captured by the camera to the host device 4, a packet for storing the image data is created. The packet creation processing will be described later. Then, in step S9, the packet created in step S8 is transmitted to the network 3 shown in FIG.

On the other hand, if it is determined in step S7 that the user's instruction is not a mail transmission or a facsimile transmission, it is determined in step S10 whether the user has pressed the save button. If the user presses the save button, the portable terminal 1 saves the image data in step S11. On the other hand, if the user has not pressed the save button, it is regarded that the user has pressed the cancel button, and the image data is discarded in step S12.

FIG. 9A is a diagram showing a structure of a packet transmitted from the portable terminal 1. This packet is created in step S8 shown in FIG. Each packet is
It consists of a header part and a data part. The header part is
Stores source address and destination address.
What address system is stored as the source address and the destination address depends on the network configuration to which the present embodiment is applied. For example, in TCP / IP communication, an IP address is stored.

The data section stores application identification information, terminal ID, image data, time information, and the like. The application identification information is information for identifying an application program to be started in the terminal (the host device 4 in this case) of the destination. That is, it is a command for activating a predetermined program in the destination terminal. In the present embodiment, information for identifying the image data analysis / recognition program is set. In addition,
The application identification information is specified as a port number in TCP / IP communication.

The terminal ID is an identification number of the portable terminal 1. The image data is image data captured by camera shooting, and is compressed when stored in a packet. Note that, as described with reference to FIG. 8, when writing with the handwriting pen is performed, the trajectory pattern of the pen input is also synthesized as image data. The time information is information indicating the time at which the image data transmitted to the host device 4 was photographed by the camera.

FIG. 9B is a detailed flowchart of the packet creation process in step S8 shown in FIG. In step S21, the image data to be transferred to the host device 4 is compressed and stored in the data section. The compression method is, for example, JP
EG. In step S22, time information indicating the time at which the image data was captured by the camera is set. In step S23, information for identifying the image data analysis / recognition program is set as application identification information.
In step S24, a terminal ID is set as information for identifying the mobile terminal 1 (information for identifying the own device). further,
In step S25, a header section is created. In the header part, at least the address of the portable terminal 1 (own address) and the address of the host device 4 are set as the source address and the destination address, respectively.

The packet created as described above is
It is sent to the network 3. The network 3 transfers the packet to the host device 4 according to the destination address of the packet. Hereinafter, the host device 4 that receives and processes this packet will be described.

FIG. 10 is a configuration diagram of the host device 4.
The storage device 51 is configured by a semiconductor memory, a magnetic recording medium, or an optical recording medium, and stores programs, data, and the like. The storage device 51 may be fixedly provided in the host device 4 or may be detachably mounted.

The recording medium driver 52 reads data stored in a portable recording medium (including a semiconductor memory, a magnetic disk, an optical disk, a magneto-optical disk, etc.) 53 or writes data into the portable recording medium 53. Device. The communication control unit 54 is a unit that controls transmission and reception of data with the network. Transmission and reception of packets with the mobile terminal 1 are also controlled here.

The CPU 55 loads a program or the like from the storage device 51 or the portable recording medium 53 into the memory 56 and executes it. Note that the program and data to be recorded in the storage device 51 may be those in which what has been stored in the portable recording medium 53 may be written, or may be transmitted from another device on the network via a communication line or the like. It may be configured to receive and record. Further, the CPU 55 may use programs, data, and the like stored in another storage device provided on the network via a communication line or the like.

FIGS. 11 to 12 are flowcharts for explaining the processing of the host device 4. Here, the operation centered on the host device 4 receiving the packet (the packet created by the processing of the flowchart in FIG. 4) transmitted from the mobile terminal 1 and processing the image data according to the information set in the data portion thereof is mainly described. Next, the operation when the host device 4 receives a packet from the network 3 will be described.

When the packet is received in step S31,
The terminal I set in the packet in step S32
By examining D, the terminal that transmitted the packet is recognized. In step S33, the received packet is
It is checked whether or not the mail data is included. If the mail data is included, it is regarded as “e-mail reception” and the process proceeds to step S41. In step S41, the mail transmission / reception unit 6 is activated to request processing related to mail reception. The mail transmitting / receiving unit 6 stores the received mail using the destination as a key, and notifies the destination user that the mail has arrived as necessary.

In step S34, it is checked whether or not the received packet contains FAX data. If the received packet contains FAX data, it is regarded as "FAX reception" and step S34 is performed.
Proceed to 42. In step S42, the fax transmission / reception unit 7 is activated to request processing related to fax reception. The FAX transmission / reception unit 7 stores the received FAX using the destination as a key, and notifies the destination user that the FAX has arrived as necessary.

In step S35, the application specified by the application identification information set in the received packet is started. Step S3
At 6, it is determined whether the activated application includes an image recognition process. If the application does not include the image recognition processing, the processing of the application is executed in step S43. If the application includes an image recognition process, decompress the image data stored in the packet received in step S37,
In step S38, pattern recognition processing and character recognition processing are performed on the image data. By this image recognition processing, the service requested by the user is recognized. The recognition processing in step S38 will be described later.

In step S51, it is checked whether the service requested by the user is "mail transmission". If it is "mail transmission", the process proceeds to step S52. Step S
At 52, pattern recognition processing and character recognition processing are performed on the image data decompressed at step S37. In addition,
In step S38, the character recognition processing is executed only for the area necessary for recognizing the service requested by the user. In step S52, the character recognition processing is executed for the entire area of the decompressed image data. Then, the character obtained by the character recognition processing is converted into a predetermined character code (for example, an ASCII code or a JIS code). The character code obtained here is a mail sentence input by the user using the mobile terminal 1. Step S53
Then, the mail transmission / reception unit 6 is activated, and the mail transmission / reception unit 6 transmits the mail text obtained in step S52. In addition,
Steps S52 and S53 will be described in detail later.

In step S54, it is determined whether or not the service requested by the user is "FAX transmission".
If “transmit”, the process proceeds to step S55. Step S5
In step 5, the facsimile transmission / reception unit 7 is activated, and the facsimile transmission / reception unit 7 is activated.
Sends the image data decompressed in step S37 to the destination specified in the image.

In step S56, it is checked whether or not the service requested by the user is "mail reading". If "mail reading", the process proceeds to step S57. In step S57, the mail transmitting / receiving unit 6 is activated. If an e-mail addressed to the terminal ID detected in step S32 has arrived, the e-mail transmitting / receiving unit 6 transmits the e-mail to the terminal I
D to the terminal having D. Step S57 will be described later.

In step S58, it is determined whether or not the service requested by the user is "FAX retrieval".
If "X take-out", the process proceeds to step S59. In step S59, the fax transmission / reception unit 7 is started. If a fax addressed to the terminal ID detected in step S32 has arrived, the fax transmitting / receiving unit 7
D to the terminal having D.

FIG. 13 is a detailed flowchart of step S38 in FIG. 11. By executing pattern recognition processing and character recognition processing on the image data stored in the received packet, the service requested by the user is provided. The processing to be detected is shown.

In step S61, a specific symbol or color is detected from the received image data according to a predetermined rule. For example, as shown in FIG. 5 (a), when a frame-shaped parenthesis is used as a symbol indicating a position where a character string specifying a service or a character string specifying a destination exists, Extract the inner image. In addition, as shown in FIG. 5B, when the service type is associated with a color and a destination is specified using a character string represented by a specific color, the specific color is used. Extract the region.

In step S62, a character recognition process is performed on the image data extracted in step S61. Then, in step S63, the service requested by the user is detected from the result of the character recognition processing. FIG.
In the example shown in (a) or (b), it is detected that the service requested by the user is "e-mail transmission", and at the same time, the destination address of the e-mail is also detected.

FIG. 14 is a detailed flowchart of step S52 in FIG. 12, and shows a process of detecting each character code of a character string transmitted as a mail text. Step S71
Then, a character recognition process is performed on the received image data, and the portable terminal 1 recognizes the captured characters. Here, the character recognition processing is not executed for the area processed in step S38. In step S72,
The position of each character detected in step S71 is detected. For example, the center position of each character is detected as a relative position with respect to the upper left corner of the screen shot by the camera.

In step S73, the pitch between characters is checked to determine whether the text is written vertically or horizontally. Here, in the case of horizontal writing, the horizontal character pitch is generally smaller than the vertical character pitch, and in the case of vertical writing, the vertical character pitch is smaller than the horizontal character pitch. Take advantage of the nature. In step S74, characters are extracted in the direction obtained in step S73. That is, for example, in the case of a horizontally written sentence, the uppermost line is extracted one character at a time from left to right, and thereafter, the same processing is sequentially performed on the lower lines. Then, these extracted characters are sequentially converted into character codes one by one and stored.

By the above-described processing, each character of a sentence fetched by the user using the portable terminal 1 is converted into a character code extracted in order. FIG. 15 is a flowchart showing step S in FIG.
52 is a detailed flowchart of a mail transmission process 52. This process is executed by the mail transmitting / receiving unit 6 in FIG.

In step S81, it is confirmed with the user of the portable terminal 1 whether the destination address obtained in step S38 is correct. That is, the destination address obtained in the image recognition processing in step S38 is replaced with the address in step S32.
Is sent to the terminal having the terminal ID detected in step (1), and an inquiry is made as to whether the destination address is correct. Mobile terminal 1
Receives the inquiry, displays the received destination address on the display unit, and waits for a user instruction.

In steps S82 and S83, a response from the portable terminal 1 is waited, and if there is no response within a predetermined time, the process ends. On the other hand, if there is a response, step S84
In S85 and S85, it is checked whether the response is "OK: Accept destination address". If "OK", the process proceeds to step S91. If a change instruction is included, the process proceeds to step S91.
At 86, the address is changed according to the instruction, and the process returns to step S81. Note that the response from the mobile terminal 1 is “O
If neither "K" nor the change instruction, step S
At 87, error processing is executed.

In step S91, a TCP segment shown in FIG. 16 is created. That is, the port number and the sequence number are set in the TCP header. The port number is, for example, a number that specifies SMTP. As the data, the character string obtained in step S52 is stored as a mail text.

In step S92, an IP address corresponding to the destination address entered in the image data is obtained. The IP address is obtained, for example, by making an inquiry to a DNS (domain name server) (not shown) existing on the network. In step S93, FIG.
Create the IP datagram shown in. IP created here
In the datagram, the IP address obtained in step S92 is set in the header, and the TCP segment is stored in the data portion.

In step S94, a transmission packet shown in FIG. 16 is created. The above IP is included in the data part of the transmission packet.
Packets are stored. The format of this packet is determined by the communication protocol on the network 3 to which the host device 4 connects. In step S95, the transmission packet created in step S94 is sent to the network 3.

In step S96, it is checked whether or not the transmission packet has been transmitted normally. If the mail has been transmitted normally, the portable terminal 1 is notified that the mail transmission has been completed in step S97, and the transmitted mail is stored in the host device 4 in step S98. On the other hand, when the transmission packet cannot be transmitted normally, the fact is notified to the portable terminal 1.

FIG. 17 is a detailed flowchart of step S57 in FIG. 12, and shows a process of reading a mail stored in the host device from the portable terminal. This process
For example, in a situation where a mail addressed to the user of the mobile terminal 1 arrives at the mail transmitting / receiving unit 6 and the mail transmitting / receiving unit 6 manages the mail, the mobile terminal 1 writes a memo as shown in FIG. Is executed when the image data is captured and the image data is sent to the host device 4.

In step S101, the user accesses the mailbox of the terminal (user) of the inquiry source. This terminal is specified by the terminal ID detected in step S32. The mail arriving at the mail transmitting / receiving unit 6 is put in a specific mailbox according to the destination. For example, when a mail addressed to the user of the mobile terminal 1 arrives, the mail transmitting / receiving unit 6 stores the mail in a mailbox provided for the user of the mobile terminal 1. Accordingly, by accessing using the terminal ID detected in step S32 as a key, the terminal (user)
You can read the letter addressed to.

In step S102, a character recognition process is performed on the received image data to check whether the sender of the mail has been designated. If there is no designation, in step S103, all the mails stored in the mailbox of the terminal are read. On the other hand, if the sender is specified, only the mail from the sender is read in step S104. In the case of the example shown in FIG.
Reads only the mail sent from Mr. K. And
In step S105, the read mail is sent to the terminal of the inquiry source.

In the above-described embodiment, an example has been described in which a certain address is specified as a destination of an e-mail. However, a configuration in which one group including a plurality of terminals is specified as a destination is provided. Is also good. In this case, it is necessary to register a table as shown in FIG. In the case where the table of FIG. 18 is provided, for example, in FIG. 5 (a), what is written as “Sales Department” instead of the destination address is photographed by the camera 12 and the image data is , Host terminal 4
Executes character recognition processing on the received image data to recognize that the destination of the mail is the “sales department”. At this time, if the table of FIG. 18 is accessed using the terminal ID of the portable terminal 1, one or more e-mail addresses belonging to the “sales department” can be obtained. Subsequent processing is as described above.

In the above embodiment, the processing program executed by the host device 4, ie, the processing program shown in FIG.
1 to 15 and 17, a program for interpreting and processing information transferred via a network, and the like are CP
U55 is stored in the storage device 51 or the portable recording medium 53 in the form of a readable program code. Alternatively, one stored in another device connected via a network is used.

Further, FIGS. 14 to 18 show the operation at the time of sending an E-mail.
The operation at the time of X transmission is the same. However, when performing FAX transmission, it is necessary to detect the telephone number of the transmission destination by executing image recognition processing on the received image data. You don't have to.

Further, in the above-described embodiment, the configuration in which the image data is captured by using the camera has been described. However, the present invention is not limited to this configuration. For example, the image data read by using the image scanner is transferred to the host device. You may make it.

As described above, the image data processing system according to the present embodiment transfers the image data captured by the portable terminal to the host device, and the host device analyzes the image data to request the user of the portable terminal. In this configuration, the service is recognized, and the transfer data included in the image data is transmitted by e-mail or FAX according to the transfer destination information included in the image data. For this reason,
Just by having a function of capturing image data and transferring it to the host device and a function of requesting the host device to perform processing, the mobile terminal can transmit an e-mail or the like with a user interface for photographing a handwritten memo or the like with a camera. become. That is, the mobile terminal does not need to execute a high-accuracy pattern recognition process and various processes that require high-speed processing. Therefore, the portable terminal can enjoy a high-level image processing result executed by the host device without providing a large-capacity memory or a high-performance processor.

[0075]

According to the system of the present invention, a camera or the like provided in a portable terminal captures a recording surface containing a desired character string as image data, and requests a necessary service from a host device, so that a handwritten memo or the like can be easily written. Email or fax
Can be transmitted to a desired party. Thus, the present invention provides a user interface with improved operability.

Since image data processing requiring high-speed processing, in particular, pattern recognition processing and character recognition are not executed on the mobile terminal side, the mobile terminal can perform advanced image data processing without providing a large-capacity memory or a high-performance processor. Processing results can be obtained.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of an embodiment.

FIG. 2 is an external view of a portable terminal.

FIG. 3 is a configuration diagram of a mobile terminal.

FIG. 4 is a flowchart of a process for requesting a desired service by transferring image data captured by a mobile terminal to a host device.

FIGS. 5A and 5B are examples (No. 1) of displaying image data captured by a camera on a display unit of a mobile terminal;

FIGS. 6A and 6B are examples (No. 2) of displaying image data captured by a camera on a display unit of a portable terminal.

FIG. 7 is an example (part 3) in which image data captured by a camera is displayed on a display unit of a mobile terminal.

FIGS. 8A and 8B are examples (part 4) of displaying image data captured by a camera on a display unit of a portable terminal;

FIG. 9A is a diagram illustrating a structure of a packet transmitted from a mobile terminal, and FIG. 9B is a detailed flowchart of a packet creation process.

FIG. 10 is a configuration diagram of a host device.

FIG. 11 is a flowchart (part 1) illustrating processing of the host device.

FIG. 12 is a flowchart (part 2) illustrating processing of the host device.

FIG. 13 is a flowchart illustrating a process of recognizing a service requested by a user from received image data.

FIG. 14 is a flowchart illustrating a process of extracting a character string of a mail text from received image data.

FIG. 15 is a flowchart illustrating a mail transmission process.

FIG. 16 is a diagram illustrating a packet storing a mail text.

FIG. 17 is a flowchart illustrating a process when a mail inquiry is received from another terminal.

FIG. 18 is a configuration diagram of a table storing email addresses belonging to each group in association with the group name.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Mobile terminal 2 Image input part 3 Network 4 Host device 5 Image recognition part 6 Mail transmission / reception part 7 FAX transmission / reception part 8 Internet 9 Terminal 10 FAX terminal 12 Camera 21 CPU 22 Storage device 24 Recording medium driver 25 Portable recording medium 31 Liquid crystal display Reference Signs List 40 communication control unit 51 storage device 52 recording medium driver 53 portable recording medium 54 communication control unit 55 CPU 56 memory

Claims (5)

[Claims] A mobile terminal for capturing image data including an image as transmission information and a character designating a transmission destination of the transmission information; and receiving the image data captured by the mobile terminal;
And an information processing device for recognizing characters included in the image data and transmitting transmission information included in the received image data to a destination device designated by the recognized characters. Processing system. 2. A mobile terminal for capturing image data including an image for specifying a mail transmission service, an image for specifying a mail transmission destination address, and an image of a character to be transmitted by mail, and an image data captured by the mobile terminal. Receive,
Recognizing the image pattern and the character of the image data, and detecting an image designating the mail transmission service, the image data includes the character to be transmitted included in the received image data. An image data processing system, comprising: an information processing device that transmits an image to a device designated by a mail destination address. 3. A portable terminal that captures an image of a mail destination address written in a predetermined color and image data including a character image to be transmitted in a color other than the predetermined color. Receiving image data captured by the mobile terminal;
The mail destination address is detected by performing the character recognition process on the image of the predetermined color, and the character recognition process is performed on the image of a color other than the predetermined color. And an information processing device for transmitting the character to be transmitted to the device specified by the mail transmission destination address. 4. A portable terminal for taking in image data including a character designating a service to which a predetermined image pattern has been given, a character designating a destination, and an image to be transmitted. Receiving image data,
Recognizing an image pattern and characters included in the image data, performing a character recognition process on the character to which the predetermined image pattern has been added to recognize a service, and recognizing the image to be transferred according to the service. An image data processing system comprising: 5. An information processing apparatus for processing data transferred from another apparatus, comprising: an image as transmission information and image data including a character designating a transmission destination of the transmission information; An image data processing apparatus comprising: a recognition unit configured to recognize a character included in data; and a transmission unit configured to transmit transmission information included in the received image data to a destination device specified by the recognized character. .