JP4953461B2 - Spam mail determination server, spam mail determination program, and spam mail determination method - Google Patents

Description

  The present invention relates to a spam mail determination server, a spam mail determination program, and a spam mail determination method for determining whether or not a transmitted electronic mail is a spam mail (spam mail).

  With the spread of electronic mail on the Internet, damage caused by so-called spam mail has become a social problem. In order to deal with such spam mail, various measures are taken on the system.

  For example, create a spam filter that records the sender's address and specific text based on spam emails sent in the past, and determine whether the email received using such spam filters is spam email How to do is known.

  However, according to the above method, the user determines whether the received e-mail is a spam mail or not, and the spam mail sender address, the predetermined text contained therein, etc. It is assumed that it is registered in the spam filter of the mail server.

  For this reason, there has been a problem that it is impossible to deal with completely new spam mail in which the sender address and the predetermined wording do not exist in the spam filter.

  In addition, a method has been proposed in which feature information is extracted from a character string included in a transmitted e-mail and a determination is made (for example, Patent Document 1). According to this method, if a predetermined character string is included in an e-mail, it may be determined that the e-mail is spam mail even if it is not originally spam mail.

JP 2006-065655 A

  As described above, various methods are known for determining spam mail, but many spam mails that are actually received are stylized, and they differ from normal mail in their visual appearance at first glance. Often, it can be determined as spam mail without looking at the contents.

  Accordingly, an object of the present invention is to provide a technique for determining whether or not a spam mail is based on a spam mail or a visual pattern of a portion included in the spam mail.

  The present inventor has found a mechanism for determining spam mail based on a visual appearance rather than the contents of the mail, and has completed the present invention. Specifically, the present invention provides the following.

  (1) Pattern recognition means for recognizing all or part of a visual pattern on which received e-mail is displayed, and all or part of visual patterns of e-mail recognized by the pattern recognition means are stored in advance A spam mail determination server comprising pattern matching means for matching with a visual pattern, and spam mail determination means for determining whether or not the electronic mail is spam mail based on a matching result of the pattern matching means.

  According to such a configuration of the present invention, since it is determined whether or not it is spam mail from the visual appearance without analyzing the content of the received electronic mail, the content is not determined as spam mail. Even an e-mail can be determined as a spam mail if the visual appearance is similar.

  In particular, since it is a determination method that has not existed until now, the determination rate of spam mail can be increased by using it in combination with an existing method.

  (2) The visual pattern of the e-mail recognized by the pattern recognition means is a visual pattern when the e-mail is displayed on a screen having a predetermined size. Spam mail judgment server.

  According to such a configuration of the present invention, since the visual appearance changes depending on the screen on which the e-mail is displayed, the invention of (1) is implemented on the premise of the screen size normally used by the user. Can do. As a result, the effect of the invention of (1) can be exhibited in a form that matches the actual situation of the user.

  (3) A spam mail determination server according to (1) or (2), wherein the stored visual pattern that is collated by the pattern collating unit differs depending on a language used in the electronic mail. .

  According to such a configuration of the present invention, since spam mail from overseas is also increasing, it is possible to classify and determine the pattern for determining spam mail according to the language. The determination accuracy can be improved.

  (4) The spam mail determination server according to any one of (1) to (3), wherein the visual pattern recognized by the pattern recognition means is recognized as one block for each sentence.

  According to such a configuration of the present invention, since a pattern is recognized as one block for each sentence, one long sentence and one short sentence are recognized as different patterns even if they have the same contents. Spam mail has relatively short texts, so it is effective for determining normal mail and spam mail.

  (5) The spam mail determination according to (4), wherein the visual pattern recognized by the pattern recognition unit recognizes a link sentence including a hyperlink and a normal sentence not including a hyperlink as different types of blocks. server.

  Spam mails usually include hyperlinks, and many of them are intended to access the links, and normal sentences often play a guiding role to access the links.

  According to such a configuration of the present invention, by performing pattern recognition that distinguishes a normal sentence and a link sentence, it is possible to determine a spam mail with the similarity of the visual appearance of the portion of the normal sentence that plays a guiding role. It is possible to effectively determine a spam mail using a guiding means having a similar appearance.

  (6) The pattern matching means has a function of determining that the block is a block related to a sentence constituting a clause based on the size of a continuous block, and when determining that the block is a clause, the sentence constituting the item The spam mail determination server according to (4) or (5), wherein the difference in the number of blocks related to is evaluated lower than the difference in the number of blocks related to a sentence that does not constitute the item.

  Even with the same type of spam mail, it is not uncommon for the number of sentences that make up the bullets included in the mail to differ. According to such a configuration of the present invention, it is determined that the item is a bullet part based on the size of successive blocks, and the difference due to the difference in the number of sentences constituting the item part is different from the difference in the number of other sentences. Since the evaluation is made lower than that, it is possible to prevent the accuracy of determining spam mails from being lowered only by the difference in the number of sentences constituting the item part.

  (7) A pattern recognition means for recognizing the whole or part of a visual pattern in which the received electronic mail is displayed, and a visual pattern of the whole or part of the electronic mail recognized by the pattern recognition means. A pattern matching unit for matching with an accumulated visual pattern, and a spam mail determination program for functioning as a spam mail determination unit for determining whether or not the electronic mail is spam mail based on a matching result of the pattern matching unit.

  According to such a configuration of the present invention, since the invention of (1) is realized using a computer by a program, the same effects as the invention of (1) can be achieved.

  (8) The visual pattern of the electronic mail recognized by the pattern recognition means is a visual pattern when the electronic mail is displayed on a screen of a predetermined size. Spam mail judgment program.

  According to such a configuration of the present invention, since the invention of (2) is realized using a computer by a program, the same effects as the invention of (2) can be achieved.

  (9) The spam mail determination program according to (7) or (8), wherein the stored visual pattern that is collated by the pattern matching unit is different depending on a language used in the electronic mail. .

  According to such a configuration of the present invention, since the invention of (3) is realized using a computer by a program, the same effects as the invention of (3) can be achieved.

  (10) The spam mail determination program according to any one of (7) to (9), wherein the visual pattern recognized by the pattern recognition means is recognized as one block for each sentence.

  According to such a configuration of the present invention, since the invention of (4) is realized using a computer by a program, the same effect as the invention of (4) can be obtained.

  (11) The spam mail determination according to (10), wherein the visual pattern recognized by the pattern recognition unit recognizes a link sentence including a hyperlink and a normal sentence not including a hyperlink as different types of blocks. program.

  According to such a configuration of the present invention, since the invention of (5) is realized using a computer by a program, the same effect as the invention of (5) can be obtained.

  (12) The pattern matching unit has a function of determining that a block recognized for the sentence is a block related to a sentence constituting a clause based on a continuous block size. The spam according to any one of (7) to (11), wherein a difference in the number of blocks related to a sentence constituting a clause is evaluated to be less than a difference in the number of blocks relating to a sentence not constituting the clause Email judgment program.

  According to such a configuration of the present invention, since the invention of (6) is realized using a computer by a program, the same effect as the invention of (6) can be achieved.

  (13) A pattern recognition step for recognizing all or part of a visual pattern on which an electronic mail received by the computer is displayed, and a visual in which the computer stores the whole or part of the visual pattern of the electronic mail in advance. A spam mail determination method comprising: a pattern matching step for matching with a pattern; and a spam mail determination step in which a computer determines whether the electronic mail is a spam mail based on a result of the comparison.

  According to such a configuration of the present invention, since the invention described in (1) is realized using a computer, the same effects as in (1) can be achieved.

  According to the present invention, the determination as to whether or not the mail is spam mail is made based on the visual appearance rather than the mail content, which is particularly effective for the determination of the standardized spam mail transmitted while changing the content. In addition, when used in combination with a conventional determination method that is determined based on the contents of an e-mail, it is possible to perform determination with higher accuracy.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

(First embodiment)
[overall structure]
FIG. 1 is a diagram showing an overall configuration of a spam mail determination server and related elements connected thereto. The spam mail determination server 1 includes an electronic mail receiving means 2, an electronic mail display content conversion means 3, a pattern recognition means 4, a pattern matching means 5, a spam mail determination means 6, a pattern dictionary DB 7, and a normal mail DB 8. And a spam mail DB 9. The pattern dictionary DB 7, the normal mail DB 8, and the spam mail DB 9 may be placed not in the spam mail determination server 1 but in an external storage device connected to the spam mail determination server 1.

  The spam mail determination server 1 is connected to the Internet 20 and the user terminal 10. There may be a plurality of user terminals 10. The spam mail determination server 1 and the user terminal 10 may be connected via the Internet, or may be connected in other forms, for example, a star network.

  The e-mail receiving unit 2 receives an e-mail transmitted via the Internet 20. The e-mail display content conversion means 3 determines whether the received e-mail is a spam e-mail as the final display content on the user terminal screen, and converts the e-mail to the display content on the user terminal screen. Convert. The pattern recognition means 4 recognizes the visual pattern contained in the e-mail converted into the display content. Usually, a plurality of visual patterns are recognized from one electronic mail. The pattern matching unit 5 matches the visual pattern registered in the pattern dictionary DB 7 with the visual pattern recognized by the pattern recognition unit 4. The spam mail determination means 6 determines whether or not the electronic mail is spam mail based on the result of matching by the pattern matching means 5 and distributes it to normal mail and spam mail. Normal mail is recorded in the normal mail DB 8, and spam mail is recorded in the spam mail DB 9. The user terminal 10 accesses the spam mail determination server 1 and requests transmission of mail addressed to itself. In response to a request from the user terminal 10, the spam mail determination server 1 reads out the mail from the normal mail DB 8 or the spam mail DB 9 and transmits it to the user terminal 10.

[Hardware configuration diagram of spam mail judgment server]
FIG. 2 is a diagram illustrating a hardware configuration of the spam mail determination server 1 according to the present embodiment. The spam mail determination server 1 includes a CPU (Central Processing Unit) 41 (a plurality of CPUs such as a CPU 42 may be added in a multiprocessor configuration), a bus line 30, a communication I / F (I / F). Interface) 43, main memory 44, BIOS (Basic Input Output System) 45, display device 46, I / O controller 47, and input devices 48 such as a keyboard and a mouse.

  The control unit 40 is a part that controls the spam mail determination server 1 in an integrated manner. By appropriately reading and executing various programs stored in the hard disk 50 (described later), the control unit 40 cooperates with the hardware described above. Various functions according to the invention are realized.

  In the communication I / F 43, the spam mail determination server 1 receives an e-mail transmission from another server or the like via the Internet 20 (FIG. 1), or receives an e-mail transmission / reception request from the user terminal 10 (FIG. 1). It is a network adapter when receiving. The communication I / F 43 may include a modem, a cable modem, and an Ethernet (registered trademark) adapter.

  The BIOS 45 records a boot program executed by the CPU 41 when the spam mail determination server 1 is started, a program depending on the hardware of the spam mail determination server 1, and the like.

  The display device 46 includes a display device such as a cathode ray tube display device (CRT) or a liquid crystal display device (LCD).

  A storage device 52 such as a hard disk 50 and a semiconductor memory 51 can be connected to the I / O controller 47.

  The input device 48 accepts input by the administrator of the spam mail determination server 1.

  The hard disk 50 stores various programs for causing the hardware to function as the spam mail determination server 1, programs for executing the functions of the present invention, tables used for executing the programs, and the like. Note that the spam mail determination server 1 can also use an external hard disk (not shown) as an external storage device.

  Although the hardware configuration of the spam mail determination server 1 has been mainly described above, the functions described above can be realized by installing a program on a computer and operating the computer as the spam mail determination server 1. . Therefore, the function realized by the spam mail determination server 1 described as an embodiment in the present invention is executed by executing the above-described method by the computer or by introducing the above-described program into the computer. This is also possible.

  Note that the computer in the present invention refers to an information processing device including a storage device, a control unit, etc., and the spam mail determination server 1 is configured by an information processing device including a storage device 52, a control unit 40, and the like. This information processing apparatus is included in the concept of the computer of the present invention.

  The control unit 40 mainly controls the pattern dictionary DB 7, the normal mail DB 8, and the spam mail DB 9 in the electronic mail receiving means 2, the electronic mail display content conversion means 3, the pattern recognition means 4, the pattern matching means 5, and the spam mail determination means 6. Corresponds mainly to the storage device 52.

  The hardware configuration of the spam mail determination server 1 has been described above. In the present invention, the user terminal 10 can also be realized with the same hardware configuration.

Incoming email
FIG. 3 is a diagram showing the contents of the received electronic mail received by the spam mail determination server 1 via the Internet 20. It consists of a header and a body, and the character code used in the e-mail is shown in the header. As indicated by a portion 100 enclosed by a square, “iso-2022-jp” is designated in the case of Japanese mail. The text 101 of the mail is shown in a portion 101 surrounded by the other square. Here, the spam mail determination server 1 functions as the electronic mail receiving means 2.

[Convert display contents to user terminal screen]
FIG. 4 is a diagram illustrating a state in which the text of the received electronic mail is converted into display content on the user terminal screen. The content of the e-mail text is finally displayed on the screen of the user terminal 10 (FIG. 1), and it is an object of the present invention to determine whether or not it is spam mail based on the visual pattern displayed there. Therefore, the spam mail determination server 1 generates display content 110 on the screen of the user terminal 10 and performs a determination process.

  In the example shown in FIG. 4, it is shown that the display content 110 after conversion includes an image. The content displayed on the screen of the user terminal 10 varies depending on the size of the user screen. For example, when the horizontal size of the mail display portion is changed, one sentence is changed from a one-line display to a two-line display. Therefore, a certain rule is determined, and display contents are converted based on the rule. Therefore, a pattern created according to the same rule is registered in a pattern dictionary described later. Here, the spam mail determination server 1 functions as the electronic mail display content conversion means 3.

[Pattern recognition]
FIG. 5 is a diagram showing how the pattern of the display information of the e-mail is recognized. In the display content 110 after conversion, the text part is divided into a normal sentence that does not include a hyperlink and a link sentence that includes a hyperlink, and is grasped as a block (a portion indicated by hatching) for each sentence. The method for recognizing the pattern is shown in the portion indicated by 120.

  In this example, since there are three normal sentences and one link sentence, the blocks corresponding to the four sentences in total are recognized. Blocks for normal sentences are indicated by diagonal hatching, and blocks for link sentences are indicated by vertical hatching. A block grasps the appearance of a sentence as a shape.

  First, the entire e-mail pattern is recognized. Next, the patterns of the text part and the image part are recognized. In the case of an e-mail without an image part, since the text part and the whole match, there is no need to recognize the pattern of the text part. Next, in the text part, a pattern of a part composed of blocks of normal sentences is recognized as a normal sentence part, and a pattern of a part composed of blocks of link sentences is recognized as a link sentence part. As described above, since many spam emails invite access to a specific hyperlink, the normal sentence part is recognized for the purpose of particularly grasping the pattern of the solicitation part.

  Note that the image unit can recognize not only the entire image as one pattern but also a pattern for a partial image. A partial image can be cut out by edge processing or the like used in image recognition.

  FIG. 6 is a diagram showing a pattern recognition method in an electronic mail having a plurality of link sentences. In this case, the normal sentence parts delimited by the link sentence part are recognized independently. This is because the normal sentence and the link sentence have different roles, and it is considered that the normal sentence preceding the link sentence represents an invitation to access a hyperlink included in the link sentence. Here, the spam mail determination server 1 functions as the pattern recognition means 4.

[Pattern Dictionary DB]
FIG. 7 is a diagram showing the pattern dictionary DB. In the pattern dictionary DB 7, patterns extracted from past spam mails and patterns created by an administrator are registered for the whole, text part, image part, and normal sentence part.

  FIG. 8 is a diagram showing a pattern dictionary prepared according to the language of the e-mail. This is because spam mail is considered to have a different pattern depending on the language in which it is created. FIG. 8 shows a Japanese mail pattern dictionary 7a, an English mail pattern dictionary 7b, and a Chinese mail pattern dictionary 7c. However, the present invention is not limited to these, and pattern dictionaries may be prepared for other languages. These pattern dictionaries are held on the pattern dictionary DB 7.

[Match pattern]
By comparing the pattern of each part grasped from the electronic mail with the pattern registered in the pattern dictionary for the part, the pattern match or similarity is determined. Each pattern is grasped as image information including a text portion and the like, and collated using a technique for judging the similarity of existing image information.

  In the similar image search technology already in practical use, visual information is indicated by numerical sequence data (image feature amount) of several hundred dimensions from images to be compared, and the degree of similarity between the data (similarity) Is evaluated as the distance between the image feature quantity vectors, that is, the difference between the image feature quantities.

  This method can be used for pattern matching of the whole and each part, but is particularly effective for pattern matching of the image part.

  Other methods are possible for pattern matching of text parts and normal sentence parts. In these portions, the degree of overlap of the block portions is important for determining the analog ratio, but the analog ratio of the block portions can be determined by the size and number of blocks. Since the block shape is determined by the block size, it is omitted. However, it is confirmed that when the similar image retrieval technique described above is used, the shape of the block can also be used as a similar factor.

  Since the block size is almost determined by the number of characters constituting the sentence, the number of characters of the sentence corresponding to the block may be used as the block size. Alternatively, the area of the block may be used. By comparing the block sizes in this way, the pattern analogy can be verified.

  FIG. 9 is a diagram illustrating an example of pattern matching of a normal sentence part. The pattern 130 extracted from the electronic mail includes five blocks. On the other hand, the pattern 140 registered in the pattern dictionary includes only four blocks. The sizes of the blocks included in the blocks are slightly different, but in any case, the size of the block 1 is large (40 in the former and 45 in the latter), and the other blocks are small in size of 10 or less. In such a case, blocks 2 and after are considered to be itemized items.

  It is normal to think that the number of items in a bullet is different even for spam emails with the same pattern. In such a case, since the number of blocks is different, it may be determined that the pattern is different. However, regarding the difference between the items that are judged as items, it is a judgment in line with the actual situation to make the judgment by reducing the weight of the analogy judgment.

  FIG. 10 shows a block size comparison table. FIG. 10A shows a comparison of block sizes between the two patterns shown in FIG. Since the blocks 1 to 4 are both the pattern extracted from the e-mail and the pattern registered in the pattern dictionary, the degree of coincidence is calculated, but the block 5 is only the pattern extracted from the e-mail. The degree of coincidence is 0. As a result, the average coincidence is 65%.

  However, as described above, since blocks 2 to 5 are considered to be item blocks, a predetermined calculation is performed on the basis of the average coincidence of the item blocks even when there is no pattern on one side. For example, when there is no pattern for one block, the matching degree adjusted as 80% (adjustment rate) of the average matching degree of the blocks in the itemized list is assigned. FIG. 10B shows the result. According to this, 62%, which is calculated by multiplying the average coincidence 78% (= (89 + 75 + 70) / 3) of the blocks in the item list by 80% of the adjustment rate, is assigned to the block 5. As a result, the average coincidence becomes 77%, and the analogy determination is performed in consideration of the clauses.

  Note that a reference block size is determined as a condition for determining whether or not the item is a block in the itemized list, and when a plurality of blocks having the block size or less continue, it is determined that the item is a itemized item. You may judge by another method.

FIG. 10C shows how to adjust the degree of coincidence when there is no one pattern for two blocks. For the first block, the adjustment rate is set to 80%, but for the second block, 60% of the average matching degree of the blocks in the itemized list (the adjustment rate of the second block) is assigned as the matching degree after adjustment. The reason why the adjustment rate is decreased in this way is that even if it is an item, when the difference in the number of blocks is large, the patterns cannot be regarded as the same.

  According to the present embodiment, the determination as to whether or not the transmitted e-mail is spam mail is based on the visual appearance rather than the contents of the e-mail, so the contents were not determined as spam mail. Spam mail that has a standard shape can be newly determined as spam mail. And since this determination process is performed by the server, this effect can be enjoyed equally by the user terminals connected to the server.

(Second Embodiment)
In the second embodiment, the processing performed by the spam mail determination server 1 in the first embodiment is performed by the user terminal 10.

[overall structure]
FIG. 11 is a diagram illustrating an overall configuration of the user terminal 10 and related elements connected thereto. The number of each element is matched with FIG. The user terminal 10 includes an electronic mail receiving means 2, an electronic mail display content converting means 3, a pattern recognition means 4, a pattern matching means 5, a spam mail determining means 6, a pattern dictionary DB 7, a normal mail DB 8, And a spam mail DB 9. The user terminal 10 is connected to the Internet 20 via the provider 150. The role of each means and each DB is the same as in the first embodiment.

  The points other than those described above are the same as those in the first embodiment.

  By doing so, the spam mail elimination process according to the present invention can be carried out also at the user terminal, so that the user who wants to prevent damage can implement the present invention individually.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

It is a figure which shows the whole structure of the spam mail determination server which concerns on 1st Embodiment, and the related element connected to it. It is a figure which shows the hardware constitutions of the spam mail determination server which concerns on 1st Embodiment. It is a figure which shows the content of the received electronic mail which the spam mail determination server 1 which concerns on 1st Embodiment received via the internet 20. FIG. It is a figure which shows a mode that the text of the received electronic mail which concerns on 1st Embodiment is converted into the display content on a user terminal screen. It is the figure which showed a mode that the pattern of the display information of the email which concerns on 1st Embodiment was recognized. It is the figure shown about the recognition method of the pattern in the email which has multiple link sentences which concern on 1st Embodiment. It is a figure which shows pattern dictionary DB which concerns on 1st Embodiment. It is a figure which shows the pattern dictionary prepared according to the language of the email which concerns on 1st Embodiment. It is a figure which shows the example of the pattern collation of the normal sentence part which concerns on 1st Embodiment. It is a figure which shows the size comparison table of the block which concerns on 1st Embodiment. It is a figure which shows the whole structure of the user terminal which concerns on 2nd Embodiment, and the related element connected to it.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Spam mail determination server 10 User terminal 20 Internet 30 Bus line 40 Control part 52 Memory | storage device 100 Display of character code 101 Received e-mail text 110 Displayed content after conversion 120 Pattern recognition figure of e-mail 130 Pattern extracted from e-mail 140 Pattern registered in the pattern dictionary 150 Provider

Claims (7)

Pattern recognition means for recognizing a visual pattern when displaying the e-mail on a screen of a predetermined size, which is the whole or a part of the received e-mail displayed;
Pattern collating means for collating a visual pattern when displaying the e-mail on a screen of a predetermined size, which is the whole or a part of the e-mail recognized by the pattern recognition means, with a pre-stored visual pattern When,
A spam mail judging means for judging whether or not the email is a spam mail based on a matching result of the pattern matching means;
A spam mail determination server comprising 2. The spam mail determination server according to claim 1 , wherein the stored visual pattern that is matched by the pattern matching unit is different depending on a language used in the electronic mail. The spam mail determination server according to claim 1, wherein the visual pattern recognized by the pattern recognition unit is recognized as one block for each sentence. The spam mail determination server according to claim 3 , wherein the visual pattern recognized by the pattern recognition means recognizes a link sentence including a hyperlink and a normal sentence not including a hyperlink as different types of blocks. When the pattern matching means has a function of determining that the block is a block related to a sentence constituting a clause based on the size of a continuous block, and determines that the block is a block related to a sentence constituting the clause The spam mail determination server according to claim 3 or 4 , wherein a difference in the number of blocks related to a sentence constituting the item is evaluated lower than a difference in the number of blocks relating to a sentence not constituting the item. Computer
Pattern recognition means for recognizing a visual pattern when displaying the e-mail on a screen of a predetermined size, which is the whole or a part of the received e-mail displayed;
Pattern collating means for collating a visual pattern when displaying the e-mail on a screen of a predetermined size, which is the whole or a part of the e-mail recognized by the pattern recognition means, with a pre-stored visual pattern When,
A spam mail determination program for functioning as a spam mail determination means for determining whether or not the electronic mail is a spam mail based on a collation result of the pattern collation means. A pattern recognition step for recognizing a visual pattern when the computer is the whole or a part of the received e-mail and displaying the e-mail on a screen of a predetermined size ;
A pattern matching step in which a computer is a whole or a part of the e-mail, and a visual pattern when the e-mail is displayed on a screen of a predetermined size is matched with a pre-stored visual pattern;
A spam mail determination step in which the computer determines whether the email is spam mail based on the result of the matching;
A spam mail determination method comprising:

Images