JP4445243B2 - Spam blocking method - Google Patents

Description

The present invention relates to a spam blocking method, in particular, it relates to a spam blocking how to cut off the delivery to spam users in the mail server.

  In recent years, when sending and receiving e-mail using communication networks such as the Internet, regardless of whether the recipient is required or not, a large number of e-mails are unilaterally sent to an unspecified number of people. It has become. Such mails are called unsolicited mails or spam mails because they give an unpleasant feeling to the recipient side and generate unnecessary packet charges.

As a countermeasure against such junk mail, a system that automatically eliminates and disposes of junk mail before the junk mail is distributed to the recipient side has been proposed (for example, see Patent Document 1).
JP 2003-131999 A (page 3-4)

The technology described in Patent Document 1 determines whether or not a URL (Uniform Resource Locator) is included in the body of an email in a mobile phone, and extracts an email containing the URL. The user refers to the e-mail stored in the trash box memory, and moves the e-mail to the e-mail storage memory when there is an unsolicited e-mail. In this way, it is possible to extract spam mail from the electronic mail distributed to the recipient side.
However, according to the above-described conventional technology, there is a problem that it is impossible to cope with a case where the URL is partially modified on the transmission side. Since the domain name can be easily updated, it is impossible to reliably block spam mail with the above-described conventional technology.

As a method for blocking spam mail, there is a technique called a rating method in addition to the technique as described in Patent Document 1. The rating method is to label each homepage on the Internet as “adult site”, “violent site”, etc., and filtering software automatically restricts access to the homepage based on those labels It is a method to do.
However, the above method has a problem that it is not possible to deal with a homepage that is not rated. In addition, this method also has a problem that it cannot cope with partial modification of the URL on the transmission side.

Furthermore, as another spam mail blocking method, there is known a method of determining a mail as a spam mail when a predetermined character is found in the body of the mail.
Currently, it is obliged to add “unsolicited advertisement *” or “! Advertisement!” To the title column of mail sent unilaterally. Therefore, when the characters “unsolicited advertisement *” or “! Advertisement!” Are found in the mail text, it is determined as a spam mail. However, since the above method is highly language dependent, it is very difficult to improve the effect in all languages.

An object of the present invention, based on the URL address included in the spam, made it possible to detect reliably spam is to provide a spam blocking how.

According to the first aspect of the present invention, there is provided a spam mail blocking method in transmission / reception of an electronic mail using a computer device , wherein the computer device converts the electronic mail into a separator character string included in a mail text. accordingly the minimum unit at which decomposes in part mail analyzing process mail structure, the computer device, an electronic fingerprint acquisition step of acquiring fingerprint for each of the decomposed part, the computing device is included in the Part A URL information acquisition step of acquiring URL information, a database reference step in which the computer device refers to a database storing electronic fingerprints or URL information acquired / accumulated from past electronic mails, and the electronic fingerprint acquisition step. The acquired electronic fingerprint or the URL information acquired in the URL information acquisition step But if it is stored as a fingerprint or URL information associated with junk mail in the database, the computer apparatus, a distribution stop step of stopping the delivery of electronic mail to determine the e-mail as spam It is solved by having provided.

  As described above, according to the spam mail blocking method of the present invention, the spam mail repeatedly delivered to the user is determined based on the electronic fingerprint of each part of the email and the URL information included in the email. Has been. For this reason, even when URL information is concealed in a nested structure part or when the URL information is camouflaged by encoding or the like, the information repeatedly sent to the user is grasped and spam mail is surely blocked. It becomes possible.

In the URL information acquisition step, a process of decoding the encoded URL character string, a process of decomposing the decoded URL character string, and a character string comprising a host name, a port number, and a path name based on the decomposed part The computer device performs a process of generating an IP address and a process of acquiring an IP address based on the host name.
By the above processing, even when URL information is forged by encoding, it is possible to detect forgery by decoding. In addition, it is possible to surely find information that is repeatedly sent by generating a “character string consisting of a host name, a port number, and a path name” that is difficult to modify or by acquiring an IP address.

Further, the construction of the database in the present invention is performed as described in claim 3.
In other words, after the database reference step, the computer apparatus, a registration processing step of performing data updating processing or the new data registration processing of said database, further comprising a, in the registration process, are acquired by the fingerprint acquisition step If the electronic fingerprint or the URL information acquired in the URL information acquisition step already exists in the database, the computer device performs a data update process for updating data and is acquired in the electronic fingerprint acquisition step. If the electronic fingerprint or the URL information acquired in the URL information acquisition step does not exist in the database, the computer device performs a new data registration process for registering as new data.

  In the data update process or the new data registration process, the number of duplicate registrations of the electronic fingerprint or URL information, the last update date and time, and the archive name of the email that first appeared are registered. Then, data is deleted from the oldest registration according to the period from the last update date to the present time.

  A step of outputting at least one of the information on the electronic mail itself, the information on the mail structure, the information on the electronic fingerprint, and the URL information as an HTML file or a text file; the HTML file or text file; If the configuration includes a step of displaying the information on the outside via the web server, the analysis result can be referred to even from the outside.

  Further, when processing for outputting the IP address as a black list applicable to a DNS server is performed, spam mail can be blocked even by a remote host by referring to the black list via the DNS.

According spam blocking how the present invention, or when the URL information in the e-mail of a multipart structure is concealed, even if the URL information is spoofed, analysis and URL email By analyzing the information, it is possible to detect these junk mails by detecting these concealments and camouflages.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The members, arrangements, and the like described below are not intended to limit the present invention and can be variously modified within the scope of the gist of the present invention.

1 is an explanatory view showing the system configuration of a general electronic mail transmission and reception, FIG. 2 is a structural view of a system spam blocking how in the embodiment of the present invention is performed, the procedure of FIG. 3 is spam detection FIG. 4 is an explanatory diagram showing a flow from mail reception to mailbox storage, FIG. 5 is an explanatory diagram showing a flow from mail reception to analysis result presentation, FIG. 6 is a list showing examples of program options, 7 is a list showing examples of actual aliases, FIGS. 8 to 10 are explanatory diagrams showing the structure of MIME mail, FIG. 11 is a flowchart showing the decryption processing of multipart email, and FIG. FIG. 13 is a list of encoded characters, FIG. 14 is an explanatory diagram illustrating examples of encoded URLs and decrypted URLs, and FIG. 15 is a spam mail. FIG. 16 is a flowchart showing the registration process in the database, FIG. 17 is a flowchart showing the process of returning the analysis result to the user, FIGS. 18 and 19 are explanatory diagrams showing an example of the analysis result, and FIG. Is an explanatory view showing the structure of the storage unit for storing the analysis result, and FIG. 21 is an explanatory view showing a specific example of characters usable in the URL.

FIG. 1 shows a system configuration for general electronic mail transmission / reception. As shown in the figure, the e-mail transmission / reception system includes a mail server 10, a user terminal 20, and a communication line network such as the Internet 30. A plurality of mail servers 10 are connected to the Internet 30, and mail transmission / reception processing is performed between the mail servers 10 .
The user terminal 20 includes mail software having a function of creating and transmitting / receiving mail. When the user terminal 20 is connected to the mail server 10, the user terminal 20 transmits the mail addressed to the user from the mailbox of the mail server 10. You can get an email. When the user terminal 20 is a mobile phone, the mail server 10 and the user terminal 20 are connected via a telecommunication line.

Figure 2 is a block diagram of a system spam blocking how in the embodiment of the present invention is performed. This system includes a mail server 10 as a server device that executes the spam mail blocking method according to the embodiment of the present invention. The mail server 10 is installed with mail server software including a spam mail blocking program and MTA (Mail Transfer Agent).
The mail server 10 is configured by a computer device such as a workstation or a personal computer (personal computer). The mail server 10 includes databases DB _{1 to} DB ₄ in which hash values of mail acquired as electronic fingerprints and URL information included in the mail are stored, databases W _{1 to} W ₄ in which white list information is stored, Data storage units 11a and 11b for storing mail analysis results, a mail box 12, a junk mail box 13, a web server 14, a POP server or an IMAP server 15 are provided. The mail server 10 also includes a control unit (not shown) that performs various processes according to MTA and various programs.

As shown in FIG. 3, in the system of this example, as registration processing in DB ₁ to DB ₄ , processing S 1 for reading an email from standard input, processing S 2 for each part element of the email, and DB reference A DB registration process S3 and a process S4 for returning the analysis result to the registered user are performed. Further, as spam determination processing, processing S1 for reading an electronic mail from standard input, processing S2 for each part element of the electronic mail, determination processing S5 by DB reference, and processing for blocking spam mail S6 are performed. .

As shown in FIG. 4, the mail server 10 receives an electronic mail transmitted from the user terminal 20 or an electronic mail transferred from another mail server. The e-mail is stored in an e-mail storage unit (not shown), but before receiving the e-mail itself, the destination of the received e-mail is determined, and the e-mail destination is local or remote. Judgment is made. If the destination is local, the electronic mail is locally processed. If the destination is remote, the email is forwarded to another server.
When the local processing is performed, the e-mail is input to the determination program, and it is determined whether or not it is a junk mail. The e-mail that is determined not to be junk mail is designated as a destination and is delivered to the mail box 12 by the local mailer program.

DB _{1 to} DB ₄ store various data necessary for determining spam mail.
DB _{1 to} DB ₄ store information extracted from electronic mail. Information stored in DB _{1 to} DB ₄ includes a URL character string (decrypted) included in an e-mail, a character string of “host: port / path” excluding a part that can be easily changed from the URL, There are four types: IP addresses obtained by DNS search for URL hosts, and hash values for each mail part. The hash value for each part of the mail is acquired as an electronic fingerprint.

Corresponding to DB _{1 to} DB ₄ , databases W _{1 to} W ₄ for white list registration are provided. In the white list registration databases W _{1 to} W ₄ , information that is known in advance to be not related to junk mail is stored.
Information to be registered in the W ₁ ~W ₄ has become the information and the pair is stored in the _DB 1 _{~DB 4,} URL string after decoding, "host: port / path" string, IP address of the mail A hash value for each part is stored. Mail corresponding to each piece of information recorded here is excluded from the junk mail determination.
DB _{1 to} DB ₄ and W _{1 to} W ₄ are in a hash format, and are “decoded URL character string”, “character string of“ host: port / path ””, “IP address”, “mail part” “Hash value” is used as a key, and “duplicate registration count”, “last update date / time (unix time)”, and “archive name of first appearing mail” are registered as data.

  The IP address is separately output to a file in the DNSBL data format. In this way, the acquired IP address can be referred to by a so-called DNSBL mechanism, and spam mail blocking based on the registered IP address can be performed even in the remote host. Recently, it is also possible to dynamically update DNS using RFC 2136, and when this technology is used, a file in a data format becomes unnecessary.

In this example, as shown in FIG. 5, it is possible to transmit an analysis result report to the user at the time of data registration / deletion operation to DB ₁ to DB ₄ or W _{1 to} W ₄ . The mail server 10 is provided with data storage units 11a and 11b in which the analysis result of URL information is stored.
Information presented by the user is formed by converting the analysis result into an HTML file. The HTML output file is stored in the data storage unit 11a.
Also, the e-mail itself is output as a text file. This information is stored in the data storage unit 11b. When a file is stored in each of the data storage units 11a and 11b, the archive name of the mail that appears first is used as the file name. The text file of the e-mail can be referred to by providing a link from the analysis result page.
The analysis results stored in the data storage units 11 a and 11 b can be viewed from the outside via the web server 14. A URL for browsing these pieces of information is transmitted by e-mail to a user who desires browsing.
The user refers to the analysis result and examines whether there is anything to be registered in the white list. If there is something to be registered in the white list, the URL information is sent by e-mail to the address for white list registration as will be described later.

The mail server 10 calls a program using alias (alias designation) for registering and deleting data in DB _{1 to} DB ₄ and W _{1 to} W ₄ and outputting data to the data storage units 11a and 11b. ing.
In this case, by transferring the mail to a specific mail address, the program written in the alias file is started with a predetermined option, and the transferred mail is input to the program by the MTA. .
FIG. 6 shows an example of options in this example. FIG. 7 shows an example of a program written in the alias file of the mail server 10.
For example, when registering URL information or the like included in junk mail in the databases DB ₁ to DB ₄ , by transferring the e-mail to the e-mail address for registration (the alias specified with the -b option) Do.
Further, when deleting specific information from the databases DB ₁ to DB ₄ , it is only necessary to write the information in a mail and send it to a mail address for deletion (alias designated with the -e option).
Further, in the case of registration in the white list databases W _{1 to} W ₄ , an e-mail is sent by writing information to be registered in the text to the e-mail address for white list registration (alias in which the −w option is specified). To do.
In the system of this example, the URL information that is the guidance destination information and the hash value of each part of the mail are used as the information for determining the junk mail, not the transmission source information. In addition, registration to the DB by simple mail transmission is possible.

DB _{1 to} DB ₄ are regularly maintained, and a DB update program is periodically executed. DB updates, the current date and time, comparing the update date and time recorded in the DB ₁ to DB _4, passed the data more than a predetermined time period is deleted. For example, “IP address” is 3 months, “Hash value of each part of mail” is 3 months, “URL string after decryption” is 1 month, “string:“ host: port / path ”” is 2 months Is deleted.

“Hash value of each part of mail”, “URL character string after decryption”, “IP address”, and “character string of“ host: port / path ”” are acquired as follows.
The “hash value of each part of mail” is obtained from each part when an electronic mail is sent as a mail composed of parts defined by MIME (Multipurpose Internet Mail Extensions).
A hash value refers to fixed-length data calculated using a hash function that projects data of an arbitrary length onto fixed-length data. The data length of the obtained hash value differs depending on the type of hash function.
In this example, a hash function called SHA1 that can obtain a 160-bit hash value is used. As the hash function, there are MD5 (128 bits), RIPEMD (160 bits), SHA256 (256 bits), etc. in addition to the above SHA1. In this example, SHA1 having sufficient practical accuracy and appropriate calculation amount is used. ing.

When the hash function is used, if the data contents are different even by 1 byte, a completely different hash value is generated. For this reason, if the hash values are the same, the data is the same, and if the hash values are different, it can be determined that the contents are different.
By using the hash value, when the same data is repeatedly sent, it can be detected and can be determined as spam mail.

When the MIME mail is MIME-multipart, it is possible to nest parts, so the analysis is performed recursively.
Here, the structure of the MIME mail will be described.
As shown in FIG. 8, the part that is the minimum unit of the MIME mail is composed of a header part, a blank line, and a body, but there is a part having a multipart structure as shown in FIGS. In this case, the header portion is described as “Content-Type: multipart / mixed”, for example.
The multi-part structure is a structure in which a separator character string is used as a delimiter and a part is further provided in the part.

9 and 10 show examples of nested multiparts. In the example of FIG. 9, one more part is provided in the text. In the example of FIG. 10, one part is provided in the text, and further parts are provided in this part. The addition of two minus signs before and after the separator character string means the end of multipart.
In addition to the nested structure, there are also a structure in which a plurality of parts are stored in parallel in a part, or a structure in which a structure stored in parallel and a nested structure are combined.
In junk mail, URLs may be concealed by including URLs in the above parts. For this reason, it is necessary to decode each part along the MIME-multipart structure, and to extract the URL included in each part.

In the mail server 10 of this example, multipart electronic mail is analyzed according to the program. FIG. 11 shows the flow of processing for decrypting a multi-part e-mail along its structure. This processing corresponds to the processing of S2 in FIG. 3 (processing for part elements).
First, a mail composed of a body and a header is passed as a part to be analyzed (step S11).
Next, the analysis of the header part of the part is performed (step S12). The header portion describes information such as the encoding method and content type.
For example, when “Content-Type: multipart / mixed” is described in the header portion, it is determined that the e-mail has a multipart structure. If the email has a multipart structure, the separator string is read.

In step S13, the body hash calculation is performed.
Furthermore, since each part may be encoded by a technique such as base64 or quoted-printable, it is determined in step S14 whether an encoding designation is described in the header part (step S14). If there is an encoding designation (step S14; Yes), the decoding process is performed in step S15, and the body is decoded. If encoding is specified but not encoded, the decoding process is not performed and the process proceeds to step S16.
If no encoding is specified (step S14; No), the process proceeds to step S16.
In step S16, it is determined whether or not the part has a multi-part structure based on the header part information analyzed in step S12.
If the structure is not a multi-part structure (step S16; No), the process proceeds to step S17 to determine whether the part is readable. If the media type of the part is readable such as text / plain, text / html (step S17; Yes), URL extraction / analysis processing is performed in step S18.
If the media type of the part is not readable (step S17; No), the process ends.

If it is determined in step S16 that the part has a multi-part structure (step S16; Yes), a process of cutting out the part inserted in the part is performed according to the separator character string (step S19). ). And the process of step S11-step S19 is performed similarly about each part cut out.
Through this process, hash values are obtained for each part constituting the e-mail, and if each part contains URL information, the information is extracted and analyzed.
As described above, in this example, the analysis procedure is recursively called in response to the fact that the MIME multipart permits a nested structure, thereby realizing a complete analysis.
The structure of the storage unit (memory) that stores the analysis results is also nested according to the MIME multipart structure. FIG. 20A shows the structure of a storage unit that stores one part. In the case of nested parts, up to 256 parts can be stored. As shown in FIGS. 20A and 20B, the included part is actually stored as a pointer to the structure shown in this definition.

Next, the URL extraction / analysis process (step S18 of the process of FIG. 11) will be described with reference to FIG.
In step S21, a search is made as to whether a predetermined scheme is included in the e-mail.
In the search of the scheme in step S21, the target URL is a URL corresponding to the following four types of specifications (schemes).
URL starting with “http: //”. This URL corresponds to HTTP (Hypertext Transfer Protocol), that is, a communication protocol for transmitting and receiving an HTML document between a web server and a web client.
URL starting with “https: //”. This URL corresponds to HTTPS (Hypertext Transfer Protocol Security), that is, a communication protocol that combines the encryption functions of HTTP and SSL (Secure Sockets Layer).
URL starting with “rtsp: //”. This URL corresponds to RTSP (Real Time Streaming Protocol), that is, a communication protocol for transferring audio data and video data in real time.
URL starting with “ftp: //”. This URL corresponds to FTP (File Transfer Protocol), that is, a file transfer protocol.
Here, when there is no URL, data in the following steps is not generated, and only the hash value is referenced.

In step S22, the end point of the URL is determined.
In step S23, it is determined whether the URL is encoded. If it has been encoded (step S23; Yes), decoding is performed in step S24, and the end point of the URL is determined again.
The processing of step S23 and step S24 is performed in order to cope with the URL camouflaged by the encoding, and “decoded URL character string” can be acquired by this processing.
Examples of encoding methods include character entity reference, numerical entity reference, and escape encoding.
The character entity reference is a technique for designating a character with a name defined by DTD (Document Type Definition; document type definition), and the character code position is surrounded by “&” and “;”.
As an example of character entity reference, for example, “&”, “&amp;” is displayed. For example, “&en;” is displayed for “¥”. For example, if “<”, “&lt;” is displayed.
In the numerical entity reference, for example, when a decimal number is specified, the character code position is enclosed by “&#” and “;”.
As an example of numerical character reference, for example, “&#38;” is displayed for “&”. For example, “&#165;” is displayed for “¥”. For example, if “<”, “&#60;” is displayed.

Escape encoding is also a framework used to capture characters that cannot be originally used in URLs, and is sometimes used for URL spoofing. The escape encoding is encoded as a triple character as described in RFC2396. This triple character is composed of a “%” character followed by two hexadecimal digits. When escape encoding is performed in the URL, “w” is displayed as “% 77”.
As described above, since a specific character is used for encoding, in step S23, it is determined whether or not the character is encoded. In the URL, “&”, “#”, “; It is checked whether or not the characters “%” and “%” are used.

In step S24, a character list is read to decode the encoded characters. In the character list, for example, as shown in FIG. 13, encoded characters and decoded characters are registered in pairs. Then, the encoded characters are decoded with reference to this list.
FIG. 14 shows an example of decoding. In the figure, the upper row is the encoded URL, and the lower row is the decoded URL. Example 1 is an example of encoding by numerical entity reference, Example 2 is an example of encoding by numerical entity reference (no semicolon), and Example 3 is an example of encoding by escape (URI-encode).

Here, the unnecessary part is cut off with characters usable in the URI. In other words, the characters that can be used are checked for each part of the extracted URL, and when an unusable character is found, it is determined as the end. FIG. 21 shows specific examples of characters used for end determination, characters usable for the user information portion, and characters usable for portions other than the user information.
In step S25, the URL is broken down into elements.
The URL represents the location of various resources used on the Internet, and the URL includes information such as a protocol, a directory, and a port for extracting the resource.
The URL is generally in the following format.
“Scheme: // userinfo @ host: port / path? Query”
Note that “userinfo @”, “: port”, “path”, and “? Query” may be omitted.
In step S25, the URL is decomposed into “scheme”, “userinfo”, “host”, “port”, “path”, and “query”.

For example, for example,
“Http: // ando: password @ ns @ www.ppml.tv /”
Is decomposed as follows.
“Userinfo” is “ando: password @ ns”.
“Host” is “www.ppml.tv”.
“Port” is omitted, but “80” because it is http. The port number is 80 for http, 443 for https, 21 for ftp, 554 for rtsp.
“Path” is “/”.
“Query” is “none”.
“Scheme” is “http”.

Some URLs described in spam mails appear to be different URLs by adding characters or the like in order to prevent them from being picked up as specific URLs.
For example, using a character string that includes "@" on userinfo purposely,
“Scheme: // intrude @ intercept @ host: port /”
There is a URL that looks like this. In this case, “intrude @ intercept” is a character string to be treated as userinfo.
Also, include “blank characters” in userinfo,
There is a URL that looks like "http: // us er in f o @ host: port /". In this case, “us er i n f o” is a character string to be treated as userinfo.
In addition, include “line feed” in userinfo,
“Http: // user (line feed)
There is a URL such as “info @ string @ hostname: port /”. In this case, except for “line feed”, “userinfo @ string” is a character string to be treated as userinfo.

Also, the host name part may be disguised.
a. “Www.ppml.tv” is changed to “www.PpMl.tv” by replacing upper and lower case letters.
b. The host name “www.ppml.tv” is the IP address “210.138.35.27”.
c. The IP address “210.138.35.27” is set to “0xD2.0x8A.0x23.0x1B” using the hexadecimal (hexadecimal number) format.

d. The IP address “210.138.35.27” is set to “0322.0212.043.03” using the Octal (octal) format.
e. The IP address “210.138.35.27” is set to “0xD28A231B” using the hexadecimal format (unsigned long: unsigned long integer).
f. The IP address “210.138.35.27” is set to “3532683315” using the Decimal format (unsigned long).

As described in a above, a capital letter such as “www.PpMl.tv” is converted to a small letter such as “www.ppml.tv”.
In addition, as in the above b to f, the IP address that has been rewritten in another format is unified with a decimal number notation such as “210.138.35.27”.

In step S26, a “host: port / path” character string is generated from the decomposition result in step S25. This character string is a part that cannot be easily changed in the elements of the URL, and can be used effectively when determining spam mail.
In the example of “http: // ando: password @ ns @ www.ppml.tv /”, “www.ppml.tv:80/” is generated as a “host: port / path” character string.

In step S27, processing for acquiring an IP address from the host name is performed.
This process is omitted when the IP address is obtained in step S25.
In step S28, when the host name “www.ppml.tv” is obtained in step S26, an IP address is acquired based on this host name. The IP address can be obtained by accessing a DNS (Domain Name System) server. All of the obtained IP addresses are acquired.

In this way, the URL extraction and analysis are completed. Even if the URL is camouflaged in various forms by the processing of step S21 to step S27, the camouflage is solved and “decoded URL character string”, “character string of“ host: port / path ””, “ IP address "can be obtained.
The analysis of the URL is finished, the analysis results are stored in the database _DB 1 to DB ₄ shown in FIG. _Thereafter, the process of the registration processing of the determination process and _DB 1 to DB ₄ by DB 1 to DB ₄ reference is made.

Here, exception processing performed after step S24 of URL extraction / analysis processing will be described. In this process, the redirector is eliminated.
In order to prevent the URL described in the spam mail from being prohibited as a specific URL, an unrelated site redirect function may be used. That is, another URL is added before the original URL, and the user who has accessed the other URL is forcibly led to a page that the user wants to originally display.

For example, the URL that illegally used the redirector is
“Http://srd.abcde.com/drst/800501378255/*http://www.365pharm1.com/”
It is described as follows.
Of the above URL, “http://www.365pharm1.com/” is a portion indicating a page that the user originally wants to show.
In step S41, the position of “*” described in the URL is determined. Next, in step S42, a process of extracting a URL (in this case, “http://www.365pharm1.com/”) described after “*” that is originally intended to be shown to the user is performed.
When the URL is extracted, the URL is analyzed in steps S26 to S28 of the URL extraction / analysis process.

Next, referring to FIG. 15, the spam mail determination process and the spam mail blocking process will be described. The determination process corresponds to step S5 and step S6 shown in FIG.
The determination flow is performed by calling a program for performing the following processing from the MTA instead of the local mailer.
When the “hash value of each part of mail”, “decoded URL character string”, ““ host: port / path ”character string”, and “IP address” are acquired through the processing of step S1 and step S2. , DB _{1 to} DB ₄ are referred to, the score of the matching data is calculated, and it is determined whether or not the score is 1 or more (step S31).
If the score is 1 or more (step S31; Yes), it is determined that the mail is spam and mail is delivered to the spam mail box 13 (step S32).
If the score is 0 (step S31; No), it is determined that it is not a spam mail, and the mail is delivered to the user's mail box 12 (step S33).
At this time, the detection result (score) is added to the mail header and delivered. By doing so, processing such as sorting using header information can be performed by MUA (Mail User Agent). It should be noted that when the delivery destination account is switched, it can be switched by using the -u and -r options in FIG.

FIG. 16 shows registration processing in the databases DB ₁ to DB ₄ . This process corresponds to step S3 shown in FIG.
The registration flow is performed by transferring an email to a registration email address. If it does so, the program for registration will be called and registration processing will be performed.
In the registration process, based on the analysis result, “the hash value of each part of the mail”, “the URL character string after decryption”, “the“ host: port / path ”character string”, “IP address” Registered in the databases DB _{1 to} DB ₄ .
In step S41, DB _{1 to} DB ₄ are referred to for the acquired four types of data, and it is determined whether or not the information is registered in DB ₁ to DB ₄ . If registration has been made (step S41; Yes), the number of duplicate registrations (counter) is incremented by 1, the update date is updated, and re-registration is performed (step S42).
When there is no registration (step S41; No), it becomes new registration. The newly registered URL information is registered with the duplicate registration count (counter) being 1, the update date / time is currently, and the archive name is currently being processed (step S43).

FIG. 17 shows a process of returning the analysis result to the user. This process corresponds to step S4 in FIG.
In step S51, the e-mail is output as a text file. The output data is stored in the data storage unit 11b with the file name “archive name.txt”.
In step S52, the analysis result is output in HTML. The output data is stored in the data storage unit 11a with the file name “archive name.html”.
In step S53, a URL for browsing these pieces of information is transmitted by mail to a user who wishes to browse the analysis results. The text file can be referred to by providing a link from the analysis result page.

18 and 19 show examples of analysis results.
In the example shown in FIG. 18, the type of sent email (symbol A in the figure), the hash value of the body (symbol B in the figure), the media type of each part (symbol C in the figure), the Hash value (codes D, E, F in the figure), number of URLs included in each part (codes G, H, I in the figure), URL character string at the time of sending (code J in the figure), URL character string after decryption (symbol K in the figure), result information obtained by decomposing the URL (symbol L in the figure), "host: port / path" character string (symbol M in the figure), IP address (symbol) Information such as N) is displayed.
In the example shown in FIG. 19, in addition to the analysis result of the sent e-mail, data registered in DB ₁ to DB ₄ is also displayed.
In FIG. 19, the type of sent email (symbol A in the figure), the hash value of the text (symbol B in the figure), the number of URL information included in the email (symbol G in the figure), the delivery URL character string at the time (symbol J in the figure), URL character string after decryption (symbol K in the figure), information obtained by decomposing the URL (symbol L in the figure), "host: port / path" characters A column (symbol M in the figure) and an IP address (symbol N in the figure) are displayed.
Furthermore, the number of duplicate registrations of the extracted IP address (symbol O in the figure), the update date (symbol P in the figure), and the archive name (symbol Q in the figure) are displayed.

In the analysis result, since the data with NEW such as “NEWURL” is newly registered, the user checks the contents.
At this time, if there is URL information that is not desired to be registered in the DB (that is, URL information irrelevant to spam mail), the URL information is registered in the white list.
In this case, URL information irrelevant to the spam mail is described in the body of the mail, and the mail is transmitted to the whitelist registration mail address. Then, the program described in “Whitelist registration and report” in FIG. 7 is started. Then, the corresponding information is deleted from the databases DB _{1 to} DB ₄ and the white list is registered in the databases W _{1 to} W ₄ .

When a log file is created, URL information included in the mail, an analysis result thereof, and a detailed score (number of hits in DB / number of detections) are recorded in the log file.
The log file is recorded in the following format, for example.
X-Picky-Score: 101 (ip: 23/27, hpp: 39/43, url: 39/43, psig: 0/3)
In the above example, all of the same IP address (ip), the same hostname: port / pth string (hpp), the same URL (url), or the same electronic fingerprint (psig) of each part on the mail server This indicates that 101 are registered in DB ₁ to DB ₄ .
In this example, for ip (IP address), 23 out of 27 detections match the information in DB ₃ , and for hpp (“host: port / path” character string), 39 out of 43 detections. Matches the information in DB ₂ , 39 out of 43 detections for url (decoded URL character string) match the information in DB ₁ , and 3 for psig (the hash value of each part of the mail) 0 of number detection is shown to match the information in the DB _4.

The system of this example may be used to restrict access to harmful websites.
Also in this case, URL information related to the harmful website is acquired in the same manner as in the above embodiment. In the terminal having the URL information, when access to the harmful website is attempted based on the registered URL information, the access is restricted.

The technical ideas other than the claims that can be grasped from the above configuration will be described below.
(1) A method for restricting access to a specific website induced by e-mail,
A mail analysis step of disassembling the electronic mail into parts that are the minimum unit of a mail structure according to a separator character string included in a mail body;
An electronic fingerprint acquisition step of acquiring an electronic fingerprint for each disassembled part;
URL information acquisition step of acquiring URL information included in the part;
A database reference process for referring to a database storing electronic fingerprints or URL information acquired / accumulated from past e-mails;
When the electronic fingerprint acquired in the electronic fingerprint acquisition step or the URL information acquired in the URL information acquisition step is stored as electronic fingerprint or URL information related to spam mail in the database, the URL information is An access restriction method for a specific website, comprising: an access restriction step for restricting access to the harmful website by determining that the information is about a harmful website.

We evaluated the functionality of this system using a site that provides data on spam (for example, spam archives; http://www.spmarchive.org/).
By analyzing the spam mails one by one and registering them as spam mails, about 37,000 mails for 34 days were registered in the DB.
As a result, it was possible to block 99.58% of spam mails with the data on the 31st day only if the IP address could be obtained from the URL. This is a detection rate that exceeds the general spam blocking method.

As a result of analyzing 37,000 mails, the data entry recorded in each DB is
URL string DB: 68,473 cases host: port / path string DB: 40,526 cases Hash value DB of each mail part: 37,424 IP address DB: 5,202 cases .
Among these, the IP address DB contributed most to the detection.
As described above, in the system of this example, high detection efficiency can be achieved with a small-sized DB, so that it can be applied to a large-scale site.
Also, in the method of judging spam mail using the sender's host, IP address, and sender's domain as in the conventional mechanism, if it is registered as the sender of spam mail, then that sender However, in the system of this example, the mail arrives as long as the destination URL information is deleted, so it can be said that the system is a safe system with less harmful effects in practice.

It is explanatory drawing which shows the system configuration | structure of a general electronic mail transmission / reception. It is a configuration diagram of a system spam blocking how in the embodiment of the present invention is performed. It is a block diagram which shows the procedure of junk mail detection. It is explanatory drawing which shows the flow from mail reception to mailbox storage. It is explanatory drawing which shows the flow from mail reception to analysis result presentation. It is a list which shows the example of the option of a program. It is a list which shows the example of an actual alias. It is explanatory drawing which shows the part used as the minimum unit of MIME mail. It is explanatory drawing which shows the MIME mail which consists of a multipart structure. It is explanatory drawing which shows the MIME mail which consists of a multipart structure. It is a flowchart which shows the decoding process of a multipart email. It is a flowchart which shows extraction / analysis processing of URL. It is a list of encoded characters. It is explanatory drawing which shows the example of encoded URL and decoded URL. It is a flowchart which shows the determination process of a spam mail. It is a flowchart which shows the registration process to a database. It is a flowchart which shows the process which returns an analysis result to a user. It is explanatory drawing which shows an example of an analysis result. It is explanatory drawing which shows an example of an analysis result. It is explanatory drawing which shows the structure of the memory | storage part which preserve | saves an analysis result. It is explanatory drawing which shows the specific example of the character which can be used for URL.

10 mail server, 11 mail receiving unit, 12 control unit, 13 storage unit, 14 mail box, 15 junk mail box, 16 remote processing unit, 20 user terminal, 30 Internet

Claims (7)

A spam mail blocking method for sending and receiving e-mail using a computer device ,
A mail analysis step in which the computer device decomposes the electronic mail into parts that are the minimum unit of a mail structure according to a separator character string included in a mail text;
An electronic fingerprint acquisition step in which the computer device acquires an electronic fingerprint for each disassembled part;
A URL information acquisition step in which the computer device acquires URL information included in the part;
A database reference step in which the computer device refers to a database in which electronic fingerprint or URL information acquired / accumulated from past e-mails is stored;
When the electronic fingerprint acquired in the electronic fingerprint acquisition step or the URL information acquired in the URL information acquisition step is stored in the database as an electronic fingerprint or URL information related to spam mail, the computer device And a delivery stop step for determining that the email is spam and stopping delivery of the email. In the URL information acquisition step, a process of decoding the encoded URL character string, a process of decomposing the decoded URL character string, and a character string comprising a host name, a port number, and a path name based on the decomposed part The spam mail blocking method according to claim 1, wherein the computer device performs a process of generating an IP address and a process of acquiring an IP address based on the host name. After the database reference step, the computer device further comprises a registration processing step of performing data updating processing or the new data registration processing of said database,
In the registration processing step, when the electronic fingerprint acquired in the electronic fingerprint acquisition step or the URL information acquired in the URL information acquisition step already exists in the database, the computer device updates the data When the electronic fingerprint acquired in the electronic fingerprint acquisition step or the URL information acquired in the URL information acquisition step does not exist in the database, the computer device registers as new data. 3. The spam mail blocking method according to claim 1, wherein new data registration processing is performed.   4. The inconvenience according to claim 3, wherein in the data update process or the new data registration process, the number of duplicate registrations of the electronic fingerprint or URL information, the last update date and time, and the archive name of the email that first appeared are registered. Mail blocking method.   5. The spam mail blocking method according to claim 4, wherein data is deleted from the oldest registration according to a period from the last update date to the present time. Outputting at least one of information on the electronic mail itself, information on the mail structure, information on the electronic fingerprint, and the URL information as an HTML file or a text file;
The method for blocking spam mail according to claim 1 or 3, further comprising a step of displaying the HTML file or text file externally via a web server.   3. The spam mail blocking method according to claim 2, wherein a process of outputting the IP address as a black list applicable to a DNS server is performed.

Images