JP2006318070A - Method for cleaning list for electronic mail and program for the same and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize effective list cleaning while preventing a reception block by an Internet service provider when transmitting large amounts of mails. <P>SOLUTION: This method for cleaning the list of an electronic mail address to be used for the Internet is provided to read an electronic mail address list, and to repeat the transmission of only a portion of electronic mails and the pause of transmission in a fixed time based on electronic mail addresses acquired from the read electronic mail address list for transmitting electronic mails, and to analyze a bounce mail for the transmission of the electronic mails, and to delete the electronic mail addresses determined as invalid addresses according to the analysis from the electronic mail address list. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method for removing invalid or unused e-mail addresses from a list of e-mail addresses, and software and apparatus for realizing the method.

[Need for cleaning]
Due to the widespread use of e-mails, e-mail advertisements and promotions have come to be used frequently. That is, based on a list (database) created by collecting a large number of e-mail addresses by various methods, advertisements, advertisements, and the like transmitted by e-mail are frequently performed.
In this way, when sending information of the same content to a large number of destinations, using e-mail, this can be achieved overwhelmingly easily and cheaply compared to the case of using a conventional paper-based mail. Can do.

On the other hand, junk e-mails to which unintended e-mails are sent have become a social problem, and the phenomenon that e-mail addresses are frequently changed to avoid this has become particularly noticeable in mobile phones. In addition, the address is often changed when the mobile carrier or the Internet provider is changed, or when the workplace or department is changed.
In this way, e-mail addresses change frequently, so even if you create an e-mail address list, if you continue to use it for a certain period of time, a significant part of its contents will be invalid addresses The content becomes obsolete.
In addition, the number of users who use long address names in recent years has increased as a countermeasure against junk mail, and in the process of collecting e-mail addresses from users, there is a high possibility that incorrect e-mail addresses will be described. Yes.

Because of the above situation, when using an e-mail address list, it is necessary to update (cleaning) such as checking the contents of the e-mail address list at regular intervals and deleting invalid addresses.
Of course, this is necessary in the sense of avoiding useless transmission and improving efficiency. Furthermore, as a recent situation, when a large amount of e-mail is sent based on a list containing many invalid e-mail addresses, this is judged as spam mail by the provider, and mail reception from the sender is blocked (reception block). There is a situation. That is, in recent years, so-called spam mails and spam mails have increased, and when a large amount of mails are transmitted in many Internet service providers (ISPs) such as mobile phone carriers, an invalid e-mail address above a certain level. A reception block is performed for a transmission source that has performed transmission including Therefore, cleaning of the e-mail address list is also required in the sense of corresponding to this.

[Problems of delivery method]
The simplest method for cleaning the e-mail address list is to remove from the list e-mail addresses that have been actually sent and then returned as unsendable. However, in order to use this method, it must actually be sent once, so even if you try to do this based on an email address list that is not frequently cleaned, it will be received by the ISP at that time. Block will be done.

As another conventional method, there is a method described in Japanese Patent No. 3315369. This is a method in which an e-mail is not transmitted but connected to the destination server by SMTP, and only the sender and destination information (envelope header) is sent and the return code returned from the destination is used for determination.
However, this method is effective when a server connected to the distribution side receives all mail and then forwards it to the inside, so that even a non-existent destination is received once. There was a problem that did not work.
In fact, a configuration employing such a gateway system has become common, and some mobile carriers have the same configuration.

[Problems of analysis method]
In order to maintain the address list of the distribution system in a healthy state with few errors, it is necessary to accurately analyze the return code and feed it back to the address maintenance.
The conventional method is expected to return RFC-compliant bounce emails, but with the recent explosion of emails, many email systems have any error, even if they are not experts. From the “consideration” to make it easier to understand what happened, error codes in Japanese and other various message return codes have been returned, and the examples that can be judged by RFC-compliant messages are better. There are few current situations.
Japanese Patent No. 3315369

As mentioned above, if mail is sent in the usual way, there is a possibility that a reception block will occur, but in order to avoid this, a method of sending only the envelope header and judging by the return code is adopted Even does not work well.
In addition, the conventional analysis method based on the assumption that an RFC-compliant message is returned does not function sufficiently at present.
The present invention has been made in view of these problems, and an object of the present invention is to provide a method for performing effective list cleaning while avoiding reception blocks by ISP, and software and apparatus for realizing the method. To do.

In order to achieve the above object, a list cleaning method according to claim 1 of the present invention is a method for cleaning a list of e-mail addresses, a procedure for reading an e-mail address list, and obtaining from the read e-mail address list A procedure for sending an e-mail by repeatedly sending a part of the e-mail address and repeating a pause for a certain period of time, a procedure for analyzing a bounce mail for the above-mentioned e-mail sending, and invalid by the analysis And a procedure for deleting the e-mail address determined to be an address from the e-mail address list.

The method according to claim 2 is the list cleaning method according to claim 1, wherein the analysis of the bounce mail includes a determination as to whether or not the bounce mail includes a predetermined error message.

The method according to claim 3, wherein the analysis of the bounce mail is compared with a file in which a keyword that characterizes an error message is patterned. It is.

Next, the program according to claim 4 is a program for cleaning a list of e-mail addresses, the process of reading the e-mail address list, and the e-mail address acquired from the read e-mail address list, Processing to send an e-mail by repeating only a part of it and suspension of sending for a fixed time, processing to analyze the bounce mail for the above e-mail sending, and the address is determined to be invalid by the analysis A program for causing a computer to execute processing for deleting an e-mail address from the e-mail address list.

The program according to claim 5 is the program according to claim 4, wherein the analysis of the bounce mail includes a determination as to whether or not the bounce mail includes a predetermined error message.

The program according to claim 6 is the program according to claim 4 or 5, wherein the analysis of the bounce mail is compared with a file in which a keyword that is a characteristic of an error message is patterned.

The apparatus according to claim 7 is an apparatus for cleaning a list of e-mail addresses, and means for reading the e-mail address list, and only a part of the e-mail addresses acquired from the read e-mail address list. A means for sending an e-mail by repeating sending and a sending pause for a predetermined time; a means for analyzing a bounce mail for the above-mentioned e-mail sending; for an e-mail address determined to be an invalid address by the analysis, A list cleaning apparatus comprising: means for deleting this from the e-mail address list.

The apparatus according to claim 8, wherein the analysis of the bounce mail includes a determination as to whether or not the bounce mail includes a predetermined error message. .

Finally, the apparatus according to claim 9 is characterized in that the analysis of the bounce mail is compared with a file in which a keyword that is a characteristic of an error message is patterned, according to claim 7 or claim 8. Device.

[Distribution method]
In an ordinary mail delivery system, “how to deliver at high speed” has been an issue. However, as described above, if a list including many invalid e-mail addresses is distributed at high speed, an ISP reception block is received. Therefore, regarding the delivery speed, in the present invention, the delivery is intentionally performed at a slower pace in order to avoid the reception block.

Since it is impossible to predict the percentage of invalid addresses included in the list to be distributed, preliminary distribution is performed using a small list randomly extracted from the distribution list and the predicted value of the distribution success rate Get. The pace at which the remaining list is distributed is adjusted according to the value.

[analysis method]
In addition, in order to appropriately perform list cleaning, the bounce mail is analyzed, and the e-mail address determined to be invalid based on the result is deleted from the e-mail address list.
Since the content of the bounce mail is not limited to the one conforming to RFC, the content is determined by comparing with the content of the pattern file collected from a lot of sample data.

According to the cleaning system of the present invention, it is possible to clean the e-mail address list in a more appropriate state while avoiding reception blocks by the Internet service provider.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an e-mail address list cleaning method and software and apparatus for realizing the same according to the present invention will be described below with reference to the drawings.

[Overview of Internet Mail]
FIG. 1 schematically shows an outline of an electronic mail propagation path on the Internet.
The mail server 1 is connected to the Internet N1, and a client 2 is connected to the mail server 1. A number of other server / client systems such as the mail server 3 and the mail server 6 are connected to the Internet N1, and Internet mail is transmitted and received between them.

The mail server 1 is a server system operating in a system such as UNIX (registered trademark). The client 2 is a computer that operates on Windows (registered trademark) or MacOS, which is generally classified as a desktop OS.
Between the mail server 1 and the client 2, the in-house system is mainly configured by a LAN, and when an individual uses Internet mail, dial-up to a server provided by an Internet provider, connection environment by ADSL, or the like is used. .
The mail server 1 accepts transmission processing from a client and transfers it to another mail server or receives a mail sent from another server, an SMTP (Simple Mail Transfer Protocol) server 1s, and a mail box for storing the received mail, It is composed of a POP or IMAP server 1p that processes a mail reception request from a client. The DNS server d provides a function called a domain name system. In the delivery process of Internet mail, the domain part of the destination address (for example, if it is an address foo@sample.co.jp, then @ To specify the IP address of the destination mail server.

SMTP is a communication protocol between a mail sending side and a receiving side, and is standardized as RFC 2821 (Request for Comments 2821).
When sending mail from the client 2 to the client 5, first, mail is sent between the client 2 and the mail server 1 by SMTP. Next, the mail server 1 queries the DNS server 1d for the destination address and obtains the IP address of the mail server 3. Next, the mail server 1 is set as the transmission side and the mail server 3 is set as the transmission destination, so that mail transmission by SMTP is performed. The mail server 3 stores the received mail in the mailbox. The client 5 can receive the mail stored in the mailbox by accessing the POP or IMAP server.

As a mail environment, a mail server 6 that only relays a mail is more often disposed outside a client 5 than an SMTP server stored in a mailbox, such as a client 9. For mail addressed to a mobile phone, the mail received by SMTP is further relayed to the communication network (non-TCP / IP) of the mobile carrier.

[Configuration of cleaning system]
FIG. 2 shows an outline of a system that performs a cleaning service, and includes a management server 11, a distribution server 12, an error analysis server 13, and a DB server 14.
The management server 11 accepts registration of the address list to be cleaned from the service use terminal 16 and stores it in the storage area of the DB server 14 or accepts an operation from the operator terminal 17.
When the service user actually uses, the list file to be cleaned is registered on the screen of the operator terminal shown in FIG. Further, the operation operator registers the contents of the mail to be distributed from the screen shown in FIG. 8 and issues an execution instruction.

The cleaning process is divided into a part for sending the distribution list 15 extracted from the storage area of the DB server 14 to the distribution server 12 and a part for receiving and analyzing the bounce mail for the sent mail by the error analysis server 13.

[Overall flow of cleaning]
A cleaning processing method will be described with reference to the flowchart of FIG.
First, the distribution list is read into the working memory area (step 301).
Then, between steps 302 to 308, the speed of mail distribution is adjusted by adopting a method of transmitting m messages from the distribution list and pausing for s seconds. At this time, the two parameters m and s are appropriately updated by feeding back the monitoring result of the ASP service.
In step 303, one address is extracted from the m items extracted from the distribution list.

Next, at step 304, the address format is checked. In step 304, a check is made as to whether the mail address has a correct format and whether a character string prohibited from being used is included.
If there is no problem in the address format check, in step 305, an SMTP connection to the mail transmission server is started. At this time, the mail transmission server for SMTP connection is the distribution server 12 of FIG.
In step 306, the mail is transmitted according to a procedure defined by RFC.
In step 307, it is determined whether m processes have been completed. If there is an unprocessed address, the process returns to step 303 to repeat the process.

When the processing for m items taken out in step 302 is completed, it is determined in step 308 whether the processing has been completed for the original distribution list. If there is an unprocessed address, the process returns to step 302 after waiting for s seconds. Take out m addresses.
In step 308, when the distribution process is completed for all addresses in the distribution list, the process ends.

Next, transmission processing of the mail received by the transmission server in step 306 will be described with reference to the flowchart of FIG.
Upon receiving the mail to be transmitted, the distribution server 12 transmits the mail according to the flow of FIG.
In step 401, DNS is inquired about the mail server of the transmission destination address, and when a normal result is not obtained in the determination in step 402, bounce mail is transmitted in step 406.
If the IP address of the destination server can be acquired, an SMTP connection is made to the partner mail server at step 403, and mail is transmitted according to the procedure specified in RFC at step 404.
If the mail transmission process at step 404 does not end normally, a bounce mail is transmitted at step 406. Here, the destination of the bounce mail is the error analysis server 13 in FIG.

The mail transmission procedure defined by RFC in step 404 will be described with reference to the flow of FIG. For convenience, the source address is userA@host.foo.jp and the destination address is
It is explained as userB@mail.sample.jp.
Connect to the TCP port 25 for the IP address already acquired from the DNS server in step (1).
If the connection is successful, a response code “220” is returned from the partner server in step (2).
In step (3), the domain name of the transmission source is notified, and if it is normally accepted, a response code “250” is returned in step (4).

In step (5), the sender's address userA@host.foo.jp is notified, and if it is successfully received, a response code “250” is returned in step (6).
Subsequently, in step (7), the destination address userB@mail.sample.jp is notified, and if it is successfully received, a response code “250” is returned in step (8).
Here, an error is returned when an address that does not exist is specified depending on the server configuration of the other party or when the mailbox of the other party's mail address exceeds the capacity limit and cannot be received. The bounce mail is transmitted to the error analysis server 13 in FIG.

In the case of SMTP processing with a server such as the SMTP gateway 22 in FIG. 2, the transmitted mail is transferred to the SMTP server 23 after being received. Therefore, the SMTP processing is performed again between the SMTP gateway 22 and the SMTP server 23, and when a bounce mail is generated, it is returned from the SMTP server 22.
When step (8) is normally completed, a signal to start data transmission is notified at step (9), and when the receiving side is ready, a response code “354” is returned at step (10).
In step (11), mail data (From, To, Subject, text, etc.) is transmitted, and an end signal is notified in step (12).
Finally, a response code “221” is returned, and the transmission process ends.

Next, a bounce mail analysis method will be described.
[Receiving route of bounce mail]
As described in the mail transmission process, the bounce mail may be returned from the distribution server 12 in FIG. 2 or may be returned from the gateway server 22 on the other side.

[Variation of bounce mail]
The bounces returned from the distribution server 12 always have a fixed configuration, but bounces returned from the partner server have various types of servers because there are various types of servers. When classified broadly, it is divided into a multi-part format in which the contents of the original mail delivered are returned in an attached file format and a single-part format quoted in the body of the bounce mail.
In addition, the error message part is not necessarily described in an RFC-compliant message. If it is written in Japanese or written in English, the specification is conscious of the arrangement of character strings. There are various things such as a line break in the middle.

[Management by pattern file]
As described above, since there are various patterns of error messages, the bounce mail analysis unit converts the keywords that are characteristic of the error messages into pattern files and manages them in the storage area.
Since there are many cases where related character strings are inserted before and after a keyword in an actual error message, management becomes complicated with simple patterning. Therefore, more error messages are processed with fewer patterns by managing them with a pattern file that uses regular expressions.

[Special processing module]
Also, bounce mail returned from some major vendor groupware and mail servers and bounce mail returned from a specific mobile carrier may require special processing as well as pattern matching. Specifically, in the bounce mail that is returned in multi-part format, what number part is to be judged, or when a line break has occurred in the middle of an error message, it is necessary to judge by a combination of two or more keywords Etc.
In order to analyze such a bounce mail, the bounce mail analysis unit employs a mechanism that determines whether special correspondence is necessary from the header part of the received bounce and calls a special processing module as necessary. ing.
In addition, this “special processing module” can be added later, and whether or not a special module that can be flexibly dealt with when a new response is required is determined from the header of the received bounce mail. Identify and determine the server type.

[Bounce email analysis flow]
The flow of bounce analysis will be described with reference to the flowchart of FIG.
In step 601, a pattern file and various setting files are read.
In step 602, the bounce mail to be analyzed is read.
In step 603, the header part is read.
In step 604, it is determined whether the bounce mail is from the outside. If the bounce mail is returned from the outside (if it is returned from other than the distribution server 12 in FIG. 2), the partner server type is selected in step 605. After the determination, it is determined in step 606 whether or not the special processing module is to be applied.
In step 607, a module corresponding to the result of step 606 is called, and conversion processing that can be applied to the processing after step 608 is performed.
After applying the special module at step 607 or when it is determined that the special module is unnecessary at step 606, the part is classified into single part and multipart at step 608, and the analysis target part is read at steps 609 and 610.
In step 611, collation with the pattern file is performed.

Keywords to be compared with the bounce message text are registered in the pattern file, and the keywords are classified for each bounce type and given a unique code.
The bounce types are classified as follows.
-TimeOut: Timeout (could not be sent within the specified time)
-MailboxFull: The capacity of the other mailbox is over.-ServerEnv: An error that depends on the environment of the other mail server has occurred.-HostUnknown: The mail server to be sent cannot be found.
・ ReceiveReject: Rejection ・ OtherFatal: Other pattern specific code consists of the code corresponding to the above bounce type, the language of the message (English, Japanese), and the serial number.
・ EN-#-00-0001
・ EN-#-01-0026

Keywords registered as patterns are registered using regular expressions, and multiple similar messages can be processed with fewer patterns. For example,
(disk)? quota. * is
over (the limitation)?
no (
such)? (mailbox | user | recipient)
It is described as follows.
In step 612, the pattern that matches the pattern is deleted from the distribution list in step 613.
As a result, the list is cleaned.

[Accurate understanding of who the error is for mail]
In the analysis of bounce mail, it is necessary to accurately grasp “who the mail is for”. However, the sender's address of the received bounce is often the system address (MAILER-DAEMON, postmaster, etc.) of the bounce email, so who should address the email address from the received bounce email? It will be necessary to judge.
To do so, either the bounce email contains the error message along with the error email address (the address of the other party you were trying to send), or the original email content is quoted including the header information. Need to be. However, there are actually cases where there is no information in the received mail. In addition, when an error that has occurred at the transfer destination is received when it is transferred from the transmitted address to another address, a bounce that an address that does not exist in the distribution list results in an error is received.
In view of this, the bounce mail analysis unit employs a method of grasping the mail addressed to the bounce received address itself.

Since bounce mail is returned to the sender address in the envelope header of the mail to be sent (the address notified in step (5) of FIG. 5), an address that has been converted to be unique for each mail destination is set at the time of delivery. is doing.
The basic idea is based on qmail's VERPS. In VERPS

bma-xxxxxxxxxx @ <source domain>
The bma part at the beginning is a reference address for receiving bounces, separated by "-", and then the destination address "@" is replaced with "=".
In order to receive all mails with such a naming convention, it is necessary to prepare a mail address corresponding to each mail, but the reference address "bma"
The problem is solved by preparing an environment that receives all the strings following "-" in a wild card manner.

However, actually
An increasing number of servers are operating with settings that do not receive from addresses that include "=". Also,
In the conversion method conforming to VERPS, when a very long address such as a recent mobile mail address is the destination, the user part of the converted address may exceed the maximum length of 64 characters defined in RFC.
So, first
In order to convert so as not to include "=", a fixed length field indicating the length of the user part is added to the head, and the user part and the domain part are sequentially connected.
For example, foo@sample.co.jp
The envelope header when sending to the

bma-foo = sample.co.jp @ <source domain>
However, in the system of this embodiment,

bma-003foohogehoge.co.jp @ <source domain>
It is said.
Also, when the user part after this conversion exceeds 64 bytes, in the present embodiment, the conversion part is shortened by further reversible conversion with a hexadecimal number.

It is a figure which shows the outline of the propagation path | route of the electronic mail in the internet. It is a figure which shows the outline | summary of the system which performs the cleaning service of this invention. It is a flowchart which shows the method of the cleaning process of this invention. It is a flowchart which shows the transmission process of the mail which the transmission server received. It is a flow which shows the mail transmission procedure prescribed | regulated by RFC. It is a flowchart which shows the flow of a bounce analysis. It is a figure which shows the example of the screen which registers the list file of cleaning object. It is a figure which shows the example of the screen which registers the list file of cleaning object.

Explanation of symbols

11 Management Server 12 Distribution Server 13 Error Analysis Server 14 DB Server 16 Service User Terminal 17 Operator Terminal

Claims (9)

A method of cleaning a list of email addresses,
To load the email address list,
A procedure for sending an e-mail by repeatedly sending only a part of the e-mail address acquired from the read e-mail address list and stopping sending for a fixed time,
Analyzing the bounce mail for the above e-mail transmission,
A procedure for deleting an email address determined to be an invalid address by the analysis from the email address list;
A wrist cleaning method comprising: The list cleaning method according to claim 1, wherein the analysis of the bounce mail includes a determination as to whether or not the bounce mail includes a predetermined error message. 3. The list cleaning method according to claim 1, wherein the analysis of the bounce mail is compared with a file in which a keyword that is a characteristic of an error message is patterned. A program for cleaning a list of email addresses,
Reading email address list,
A process for sending an e-mail by repeatedly sending only a part of the e-mail address acquired from the read e-mail address list and stopping sending for a certain period of time;
Processing to analyze the bounce mail for the above e-mail transmission,
Processing for deleting the email address determined to be an invalid address by the analysis from the email address list;
A program that causes a computer to execute. The program according to claim 4, wherein the analysis of the bounce mail includes a determination as to whether or not the bounce mail includes a predetermined error message. 6. The program according to claim 4, wherein the analysis of the bounce mail is compared with a file in which a keyword that is a characteristic of an error message is patterned. A device for cleaning a list of email addresses,
Means to read the email address list,
Means for sending an e-mail by repeatedly sending only a part of the e-mail address acquired from the read e-mail address list and stopping sending for a fixed time;
Means for analyzing the bounce mail for the above e-mail transmission;
Means for deleting from the e-mail address list an e-mail address determined to be an invalid address by the analysis;
A wrist cleaning device comprising: The list cleaning apparatus according to claim 7, wherein the analysis of the bounce mail includes a determination as to whether or not the bounce mail includes a predetermined error message. 9. The list cleaning apparatus according to claim 7, wherein the analysis of the bounce mail is compared with a file in which a keyword that is a characteristic of an error message is patterned.