JP7175244B2 - Classification device, learning device, classification method and program - Google Patents

Description

The present invention relates to a device for detecting hostile text.

Conventionally, classification systems based on machine learning have been utilized, and attacks against such classification systems using hostile text to induce misclassification are known. The attacker uses computer-generated adversarial text while preserving the original meaning. Such adversarial text fools AI systems such as Q&A, sentiment analysis, etc., and induces misclassification, eg changing results from negative to positive labels.
Therefore, a method for detecting hostile computer-generated text in text input to a classification system is desired. For example, Non-Patent Documents 1-7 propose methods for detecting computer-generated text.

Hoang-Quoc Nguyen-Son, Tran Phuong Thao, Seira Hidano, and Shinsaku Kiyomoto, "Detecting machine-translated paragraphs by matching similar words". In: the 20th International Conference on Computational Linguistics and Intelligent Text Processing (CICLING), 2019. Hoang-Quoc Nguyen-Son, Huy H.; Nguyen, Ngoc-Dung T.; Tieu, Junichi Yamagishi, and Isao Echizen, "Identifying computer-generated paragraphs using coherence and fluency features". In: the 32nd Pacific Asia Conference on Language, Information and Computation (PACLIC), 2018. Hoang-Quoc Nguyen-Son and Isao Echizen, "Detecting computer-generated text using fluency and noise features". In: the 15th International Conference of the Pacific Association for Computational Linguistics (PACLING), pp. 288-300, 2017. Roee Aharoni, Moshe Koppel, and Yoav Goldberg, "Automatic detection of machine translated text and translation quality estimation". In: the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 289-295, 2014. Hoang-Quoc Nguyen-Son, Ngoc-Dung T.; Tieu, Huy H.; Nguyen, Junichi Yamagishi, and Isao Echizen, “Identifying computer-generated text using statistical analysis”. In: the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 123-130, 2017. Cyril Labbe and Dominique Labbe, "Duplicate and fake publications in the scientific literature: how many SCIgen papers in computer science?". In: Scientific, Springer, volume 94, issue 1, pp. 379-396, 2013. Yitong Li, Rui Wang, and Hai Zhai, "A machine learning method to distinguish machine translation from human translation". In: the 29th Pacific Asia Conference on Language, Information and Computation (PACLIC), pp. 354-360, 2015.

However, all conventional detection methods are limited in detectable text, and it has been difficult to accurately detect powerful adversarial text using recent deep learning.

An object of the present invention is to provide a classifying device, a learning device, a classifying method, and a program capable of accurately detecting hostile text.

The classification device according to the present invention includes: a tagging unit that divides document data into words and assigns part-of-speech tags to each of the words; a distance calculation unit that calculates the distance between the word vectors, and a feature amount calculator that calculates, as one feature amount, a statistic amount related to the distance between the word vectors for each combination of the document data and the part-of-speech tag. and a classifying unit for outputting a classification result using a model learned from document data that receives the feature quantity as input and is pre-labeled as to whether or not it is hostile text.

The feature amount calculation unit may calculate, as one of the feature amounts, a statistic excluding a value exceeding a threshold among the distances between the word vectors.

The classification device includes a frequency acquisition unit that acquires an appearance frequency in a corpus of each word included in the document data. quantity may be calculated as one of the feature quantities.

The feature amount calculation unit may calculate, as one of the feature amounts, the number of occurrences of phrases in which the same word continues in the document data for each length of the phrase.

The feature amount calculation unit may calculate the length of a sentence in the document data as one of the feature amounts.

The feature amount calculation unit may calculate an N-gram in the document data as one of the feature amounts.

A learning device according to the present invention includes: a tagging unit that divides document data into words and assigns part-of-speech tags to each of the words; a distance calculation unit that calculates the distance between the word vectors, and a feature amount calculator that calculates, as one feature amount, a statistic amount related to the distance between the word vectors for each combination of the document data and the part-of-speech tag. and a learning unit that receives the feature quantity as an input and generates a model that has learned whether or not the text is pre-labeled adversarial text.

The learning unit may generate a plurality of models using a plurality of learning algorithms, and select a model with the highest evaluation regarding output accuracy of the segment.

A classification method according to the present invention includes a tagging step of dividing document data into words and assigning part-of-speech tags to each of the words; a feature quantity calculation step of calculating a statistic relating to the distance between the word vectors as one feature quantity for each combination of the document data and the part-of-speech tag; and a classification step of outputting a classification result using a model learned from document data that is pre-labeled as to whether or not it is hostile text using the feature quantity as an input.

A classification program according to the present invention is for causing a computer to function as the classification device.

A learning program according to the present invention is for causing a computer to function as the learning device.

According to the present invention, hostile text can be detected with high accuracy.

FIG. 4 illustrates an example of hostile text in comparison to human-generated text; It is a figure which shows the functional structure of the classification device in embodiment. It is a figure which illustrates the kind of part-of-speech tag in embodiment. FIG. 4 illustrates tagging and inter-word distance for human-generated text in an embodiment; FIG. 4 illustrates tagging and inter-word distance for adversarial text in an embodiment; FIG. 5 is a diagram illustrating results of distributing distances between words to groups for human-created text in the embodiment; FIG. 5 is a diagram illustrating the result of distributing distances between words to groups for adversarial text in an embodiment; FIG. 10 is a diagram illustrating results of calculating an average and a variance as feature amounts related to word similarity for texts created by humans according to the embodiment; FIG. 10 is a diagram illustrating the result of calculating the mean and variance as a feature amount related to word similarity for hostile texts according to the embodiment; FIG. 4 is a diagram illustrating the frequency of occurrence of each word used within the corpus for human-generated text in an embodiment; FIG. 5 is a diagram illustrating the frequency of occurrence of each word used in the corpus for adversarial text in an embodiment; FIG. 5 is a diagram illustrating the results of distributing word frequencies to groups for human-generated text in an embodiment; FIG. 5 is a diagram illustrating the results of distributing word frequencies to groups for adversarial text in an embodiment; FIG. 10 is a diagram illustrating results of calculating an average and a variance as feature amounts relating to word frequency for texts created by humans according to the embodiment; FIG. 10 is a diagram illustrating the result of calculating the mean and variance as a feature quantity related to word frequency for hostile texts in the embodiment; It is a figure which shows the result of the evaluation experiment of the classification method in embodiment.

An example of an embodiment of the present invention will be described below.
The classification method in this embodiment classifies computer (machine)-generated adversarial texts and human-generated texts.

In this embodiment, the following features of adversarial text are utilized. That is, adversarial text has the same meaning as human-generated text, but human-generated text generally uses more appropriate and understandable language than adversarial text.

FIG. 1 is a diagram illustrating adversarial text compared to human-generated text.
Note that the underlined portions of the two illustrated texts indicate the characteristic points when compared with each other. These two texts are interpreted as having the same meaning, but the result of classification by sentiment analysis using AI is that human-made text A is classified as “positive,” while hostile text B is classified as “negative.” be.

Here, the words "features" and "movie" in Text A are synonymous, although they are more contextually appropriate than the words "attributes" and "film" in Text B, respectively. Also, adversarial texts tend to use simpler tenses than human-made texts. For example, "has balanced" (present perfect tense) in Text A is more complex than "balances" (present tense) in Text B, but is a better representation. Human-generated text has become more coherent through the use of appropriate words and tenses.
Such features are learned from word similarity, which will be described later.

Also, the phrases "fit their roles" and "the plot come to life" in Text A are more likely to be used by humans than the phrases "have good jobs" and "the story come alike" in Text B. On the other hand, common phrases found in adversarial texts are also frequently used in other common sentences.
Such features are learned from word frequencies, which will be described later.

In addition, adversarial texts may contain phrases with duplicated words, such as "very very" in text B, due to optimization problems in deep learning.
Such features are learned from the degree of redundancy of words, which will be described later.

FIG. 2 is a diagram showing the functional configuration of the classification device 1. As shown in FIG.
The classification device 1 (classification device, learning device) is an information processing device (computer) such as a server device or a personal computer, and includes a control unit 10 and a storage unit 20, as well as various data input/output devices and communication devices. .

The control unit 10 is a part that controls the entire classification device 1, and implements each function in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. FIG. The control unit 10 may be a CPU.

The storage unit 20 is a storage area for various programs and various data for causing the hardware group to function as the classification device 1, and may be a ROM, RAM, flash memory, hard disk drive (HDD), or the like. Specifically, the storage unit 20 stores dictionary data 21, a corpus 22, a learning model 23, etc., in addition to a program for causing the control unit 10 to execute each function of the present embodiment.

The control unit 10 includes an input unit 11 , a tagging unit 12 , a distance calculation unit 13 , a frequency acquisition unit 14 , a feature amount calculation unit 15 , a learning unit 16 and a classification unit 17 . Using these functional units, the control unit 10 generates a learning model 23 that classifies hostile texts created by machines and texts created by humans, and classifies new document data as hostile texts. , and output the results by classifying them as human-generated text.

The input unit 11 receives input of document data (text) as training data for the learning model 23 or classification targets by the learning model 23 .

The tagging unit 12 divides each of the input document data into words and gives part-of-speech tags to each of these words.
Existing morphological analysis methods can be used to add part-of-speech tags. At this time, the dictionary data 21 in which the part of speech corresponding to the language such as Japanese or English is defined is referred to.
Note that the dictionary data 21 may be stored in a device other than the classification device 1, or may be stored in a public database accessible via the Internet or the like.

FIG. 3 is a diagram illustrating types of part-of-speech tags.
Here, 45 kinds of parts of speech (POS tags) given to decomposed words and their meanings are shown, taking the case of processing English document data as an example.
The tagging unit 12 assigns one of these 45 types of part-of-speech tags to each of a plurality of input document data for all words forming the document data. For example, the nouns "Christmas", "movie", etc. included in the above text A are given the part-of-speech tag "NN", and the adverb "Almost" is given the part-of-speech tag "RB".

The distance calculation unit 13 calculates the distance between word vectors defined for each word for each word combination for each document data.
Multi-dimensional (e.g., 300-dimensional) word vectors unique to each word are learned based on large datasets, and data in which semantic similarity or relevance is indicated by the relative closeness between words. be. Word vectors are stored in the corpus 22 in advance.
The corpus 22 may be stored in a device other than the classification device 1, or may be stored in a public database accessible via the Internet or the like.

The distance calculator 13 calculates the Euclidean distance d by, for example, the following formula. Here, p and q denote two words, and p _i and q _i denote the i (1≤i≤n) element of the n-dimensional word vector.

The frequency acquisition unit 14 acquires the appearance frequency, for example, the number of appearances, of each word included in the document data in the corpus 22 .
Note that the frequency acquisition unit 14 may refer to a corpus different from the corpus 22 to which the distance calculation unit 13 refers.

The feature amount calculation unit 15 calculates the following feature amounts (1) to (5) as inputs to the learning model 23 .

(1) Word Similarity The feature amount calculation unit 15 calculates a statistic regarding the distance between word vectors as a word similarity, which is one of the feature amounts, for each combination of document data and part-of-speech tags.
The statistic may include at least one of mean or variance calculated by the following formula, for example. Here, a _i (1≦i≦n) indicates the i-th data of n data included in the group.

(2) Word Frequency The feature amount calculation unit 15 calculates a statistic amount related to the appearance frequency in the corpus 22 as the word frequency, which is one of the feature amounts, for each document data and part-of-speech tag.
Statistics, like word similarity, may include mean and/or variance.

(3) Word Redundancy The feature amount calculation unit 15 calculates the number of occurrences of phrases in which the same word continues in the document data as one feature amount for each phrase length (for example, 1 to 4). is calculated as the degree of duplication of the word.

(4) Length of Sentence The feature amount calculation unit 15 calculates the length of a sentence in document data as one of the feature amounts.

(5) N-gram The feature amount calculation unit 15 calculates N-grams in the document data as one of the feature amounts.

The learning unit 16 receives these feature amounts as input and generates a learning model 23 for classifying whether the document data (training data) is pre-labeled hostile text or not.
A method for generating the learning model 23 may be appropriately selected from various learning algorithms such as logistic regression, linear classifier, support vector machine by stochastic gradient descent, and support vector machine by iterative minimum problem optimization.
In addition, the learning unit 16 may generate a plurality of learning models using a plurality of learning algorithms. In this case, a model with high output accuracy, a model that achieves both high accuracy and low error rate, and the like. The highest learned model 23 may be selected.

When the document data to be classified is input, the classification unit 17 receives the feature amount calculated by the feature amount calculation unit 15 and outputs the classification result by the learning model 23 .

Next, a detailed description will be given of a method for calculating each of the above feature amounts.
It should be noted that existing techniques can be applied to the sentence length, for example, as proposed in Non-Patent Document 7, and to N-grams, as proposed in Non-Patent Documents 3 and 4, for example.

[Word similarity]
The words in the human-generated text are better used than the words in the adversarial text, so the similarity of the words in the texts differs between the two.

FIG. 4A is a diagram illustrating tagging and inter-word distances for human-generated text.
First, the tagging unit 12 gives part-of-speech tags to words in the text. Note that tagged labels are indicated by subscripts for each word in the figure.
Subsequently, the feature amount calculation unit 15 extracts combinations of words from the text and calculates Euclidean distances indicating similarities between the words. For example, the distance 1.2 of the word pair "Almost _RB - Chirstmas _NN " in the human-generated text A is closer than the distance 3.4 of the word pair "Almost _RB - is _VBZ ", and therefore more similar. means

FIG. 4B is a diagram illustrating tagging and inter-word distance for adversarial text.
In hostile text B, the distance between words is generally longer and the similarity between words is lower than in human-generated text A (FIG. 4A).

Here, a higher similarity indicates that a more natural word is used. may be eliminated as noise when calculating the feature amount.
In this case, for example, if we choose α=3.0, then distances greater than 3.0, shown in strikethrough, are deleted, as shown in FIGS. 4A and 4B. The remaining distance after deletion is also distributed over larger values for hostile text B overall.
When the Euclidean distance is used, the accuracy can be expected to improve as the value of the threshold value α increases, but the accuracy may decrease when the value exceeds a certain value. Since this threshold differs depending on the data set, it is preferable to search for the optimum value during learning.

Next, the feature amount calculation unit 15 distributes the calculated distances to groups for each combination of part-of-speech tags between words.
5A and 5B are diagrams illustrating the results of distributing inter-word distances into groups for human-generated text and adversarial text, respectively.

For example, a distance of 1.2 for the word pair "Almost _RB -Christmas _NN " is in the group for the part-of-speech pair "RB-NN", and a distance of 2.7 for the word pair "is _VBZ -movie _NN " is in the group for the part-of-speech pair "VBZ- NN", the distance 2.9 of the word pair "Almost _RB -attributes _NNS " is distributed to the group of part-of-speech pairs "RB-NNS", respectively.

Then, the feature amount calculation unit 15 calculates statistics such as average and variance as normalized feature amounts in each group.
6A and 6B are diagrams exemplifying the results of calculating the mean and variance as feature amounts relating to word similarity for human-created text and adversarial text, respectively.

[word frequency]
Adversarial texts often choose common words that are frequently used in common sentences, rather than idiomatic phrases or trendy phrases. Therefore, there is a difference in the frequency of occurrence of words used in the human-generated text and the adversarial text within the corpus 22 .

7A and 7B are diagrams illustrating the frequency of occurrence within the corpus 22 of each word used for human-generated text and adversarial text, respectively.
The appearance frequency may be, for example, the number of appearances in the corpus 22 . Compared to human-generated text A (FIG. 7A), the frequency of each word is generally higher for adversarial text B (FIG. 7B).

After calculating the frequency of appearance of each word contained in the input text within the corpus 22, the feature amount calculation unit 15 distributes these values to groups for each part-of-speech tag.
8A and 8B illustrate the results of distributing word frequencies into groups for human-generated text and adversarial text, respectively.

From the human-generated text A, for example, the frequency 215 of "Christmas _NN " and the frequency 327 of "movie _NN " are distributed to the "NN" group, and the frequency 572 of "is _VBZ " is distributed to the "VBZ" group. . Also from adversarial text B, for example, frequency 672 of "balances _VBZ " is distributed to the "VBZ" group.

Then, the feature amount calculation unit 15 calculates statistics such as average and variance as normalized feature amounts in each group.
9A and 9B are diagrams exemplifying the results of calculating the mean and the variance as feature amounts related to word frequency for human-created text and adversarial text, respectively.

[Word Redundancy]
When adversarial text is machine-generated, it may generate duplicate phrases of identical words that are not found in human-generated text. Therefore, there is a difference in word redundancy between the human-generated text and the adversarial text.

For example, in the aforementioned hostile text B, there is a phrase "very very" in which two words "very" are consecutive.
The feature amount calculation unit 15 calculates the number of such phrases as a feature amount for each length of the phrase, that is, the number of continuations of the same word. For example, the number of phrases is calculated for each of the consecutive numbers 1 (no duplication) to the consecutive number 4.

FIG. 10 is a diagram showing the results of an evaluation experiment of the classification method according to this embodiment.
In the evaluation experiment, first, a public data set <https://nlp. stanford. edu/sentiment/trainDevTestTrees_PTB. zip> to extract human-generated text. In addition, the existing system <https://github. <https://nlp. stanford. edu/sentiment/code. html> were selected.

Also, in the evaluation experiment, the following five types of feature amounts were adopted.
・Word similarity:
Average and variance values of Euclidean distances between word vectors calculated for each of 1035 groups, which are the number of combinations of 45 types of part-of-speech tags. As the corpus 22 from which word vectors are obtained, "GloVe"<https://nlp. stanford. edu/projects/glove/> was used.
・Word frequency:
Values of the average and variance of word appearance counts calculated for each of the 45 types of part-of-speech tags. As the corpus 22 for obtaining the number of appearances, "Web1T 5-gram"<https://catalog. ldc. upen. edu/LDC2006T13> was used.
・Word duplication:
The number of phrase occurrences calculated for each phrase length of 1 to 4.
・Sentence length:
The feature amount proposed in Non-Patent Document 7 mentioned above.
- N-gram The feature value proposed in Non-Patent Documents 3 and 4 mentioned above.

The results of the evaluation experiment show the classification accuracy and equal error rate (EER) when three types of supervised learning are performed using these data.
For comparison, the results of the conventional method proposed in Non-Patent Document 1 are also shown.
According to the results of this evaluation experiment, the proposed method of this embodiment includes logistic regression (learning method 1), support vector machine by stochastic gradient descent (learning method 2), and support vector machine by iterative minimum problem optimization method. In any of (learning method 3), improvements in accuracy and equivalent error rate were observed compared to the conventional method.

According to the present embodiment, the classification device 1 assigns part-of-speech tags to words forming document data, and calculates the statistic of the distance between words for each combination of part-of-speech tags as a feature amount. The classification device 1 then generates a learning model 23 based on a known classification of whether or not the text is adversarial text, using this feature quantity as an input.
Therefore, the classification device 1 can generate an appropriate learning model 23 by representing the similarity or relationship tendency of different words between the text created by a human and the hostile text as a feature amount related to the degree of word similarity.
As a result, the classifier 1 can accurately detect adversarial text that causes misclassification while preserving the meaning of the sentence. As a result, inconvenient and hostile texts can be excluded with high accuracy, for example, when product reviews are analyzed or chatbots analyze questions/answers.

The classification device 1 excludes values exceeding a threshold among the distances between words from the calculation of the statistic.
Therefore, the classification device 1 can prevent a combination of words whose similarity is less than a certain value, that is, a combination of words with low relevance, from being included in the feature amount, and can improve the detection accuracy of the hostile text.

In addition, the classification device 1 calculates, as a feature amount for each part-of-speech tag, a statistic of the frequency of appearance of the words forming the document data in the corpus 22 .
Therefore, the classification device 1 can generate an appropriate learning model 23 by representing the usage tendency of different words in the human-generated text and the hostile text by the feature amount related to the word frequency. It can detect text with high accuracy.

In addition, the classification device 1 calculates the number of occurrences of phrases in which the same word continues in the document data as a feature amount for each length of the phrase.
Therefore, the classification device 1 can generate an appropriate learning model 23 by representing the tendency specific to the hostile text as a feature amount related to the word redundancy, and as a result, can accurately detect the hostile text.

In addition, the classification device 1 can generate a more appropriate learning model 23 by combining feature quantities used in existing methods, such as sentence length or N-grams, and as a result, can accurately detect hostile text. .

Furthermore, the classification device 1 generates the learning model 23 using a plurality of learning algorithms and selects the one with the highest accuracy evaluation, so that the hostile text can be detected with higher accuracy.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Moreover, the effects described in the above-described embodiments are merely enumerations of the most suitable effects produced by the present invention, and the effects of the present invention are not limited to those described in the embodiments.

This embodiment has been described on the assumption that the training data and the document data to be detected are text of one sentence, but the document data may consist of a plurality of sentences. In this case, the classification device 1 may calculate and learn or classify the feature quantity for each sentence, or may calculate and learn or classify the feature quantity for the entire sentence.

In addition, in the present embodiment, the number of appearances is used as a feature quantity for the degree of duplication of words. A statistic averaged according to the number or length of , may be used as a feature.

In this embodiment, English text is exemplified as document data, but the target is not limited to this, and can be similarly applied to other languages including Japanese.

A learning method and a classification method by the classification device 1 are realized by software. When it is implemented by software, a program constituting this software is installed in an information processing device (computer). Further, these programs may be recorded on removable media such as CD-ROMs and distributed to users, or may be distributed by being downloaded to users' computers via a network. Furthermore, these programs may be provided to the user's computer as a web service through the network without being downloaded.

1 classification device 10 control unit 11 input unit 12 tagging unit 13 distance calculation unit 14 frequency acquisition unit 15 feature amount calculation unit 16 learning unit 17 classification unit 20 storage unit 21 dictionary data 22 corpus 23 learning model

Claims (11)

a tagging unit that divides document data into words and assigns part-of-speech tags to each of the words;
a distance calculation unit that calculates, for each of the document data, a distance between word vectors defined for each of the word combinations for each of the word combinations;
a feature quantity calculation unit for calculating, as one feature quantity of the document data, a statistic relating to the distance between the word vectors distributed to the groups for each combination of the part-of-speech tags for each of the document data ;
a classifying unit that receives the feature amount as an input and outputs a classification result by a model learned from document data that is pre-labeled as to whether or not it is hostile text. 2. The classification apparatus according to claim 1, wherein the feature amount calculation unit calculates, as one of the feature amounts, a statistic amount excluding a value exceeding a threshold among the distances between the word vectors. A frequency acquisition unit that acquires the frequency of occurrence of each word in the corpus included in the document data,
3. The feature amount calculation unit according to claim 1, wherein the feature amount calculation unit calculates, as one of the feature amounts, a statistic related to the appearance frequency distributed to the groups for each of the part-of-speech tags for each of the document data. Classifier. 4. The feature amount calculation unit according to any one of claims 1 to 3, wherein the feature amount calculation unit calculates, as one of the feature amounts, the number of occurrences of phrases in which the same word is repeated in the document data for each length of the phrase. Classification device according to claim 1. 5. The classification apparatus according to any one of claims 1 to 4, wherein the feature amount calculation unit calculates the length of a sentence in the document data as one of the feature amounts. 6. The classification apparatus according to claim 1, wherein said feature amount calculation unit calculates N-grams in said document data as one of said feature amounts. a tagging unit that divides document data into words and assigns part-of-speech tags to each of the words;
a distance calculation unit that calculates, for each of the document data, a distance between word vectors defined for each of the word combinations for each of the word combinations;
a feature quantity calculation unit for calculating, as one feature quantity of the document data, a statistic relating to the distance between the word vectors distributed to the groups for each combination of the part-of-speech tags for each of the document data ;
a learning unit that receives the feature amount as an input and generates a model that has learned classification as to whether or not the text is pre-labeled adversarial text. 8. The learning device according to claim 7, wherein the learning unit generates a plurality of models using a plurality of learning algorithms, and selects a model with the highest evaluation regarding output accuracy of the division. a tagging step of dividing the document data into words and assigning part-of-speech tags to each of the words;
a distance calculation step of calculating a distance between word vectors defined for each of the word combinations for each of the document data;
a feature quantity calculating step of calculating , for each of the document data, a statistic relating to the distance between the word vectors distributed to groups for each combination of the part-of-speech tags, as one of the feature quantities of the document data ;
A classification method in which a computer executes a classification step of outputting a classification result by using a model learned from document data that is pre-labeled as to whether or not it is adversarial text, using the feature quantity as an input. A classification program for causing a computer to function as the classification device according to any one of claims 1 to 6. A learning program for causing a computer to function as the learning device according to claim 7 or 8.

Images