CN107220667A - Image classification method, device and computer-readable recording medium - Google Patents

Abstract

The disclosure relates to an image classification method, device and computer-readable storage medium, and belongs to the technical field of image processing. The method includes: determining the features of the target image through the CNN model, and processing the features of the target image through the RNN model to obtain multiple labels of the target image. Since the features of the target image are output by the last pooling layer included in the CNN model, that is, after the CNN model obtains the features of the target image, it does not determine the probability of each category that the target image belongs to according to the features of the target image, and The method is to directly process the features of the target image through the RNN model to obtain multiple labels of the target image, so as to classify images including at least two types of content.

Description

Image classification method, device and computer-readable storage medium

technical field

The present disclosure relates to the technical field of image processing, and in particular to an image classification method, device and computer-readable storage medium.

Background technique

When a computer receives an image, it usually needs to classify the image to determine the category to which the image belongs, and set the category to which the image belongs as a label of the image, so that users can easily find the image. For example, when the computer receives a landscape image, it can determine that the category of the image is landscape, that is, set the label of the image as landscape.

In the related art, the classification of images by computers is mainly realized by a single-label classification method, that is, for multiple preset categories, at least one convolutional layer included in a CNN (Convolutional Neural Networks, Convolutional Neural Network) model and at least one pooling layer to determine the feature of the image, determine the probability that the image belongs to each category in the plurality of categories according to the feature of the image through the fully connected layer included in the CNN model, and determine the category with the greatest probability as The category to which the image belongs to get the label of the image.

Contents of the invention

In order to overcome the problems existing in related technologies, the present disclosure provides an image classification method, device, and computer-readable storage medium. The technical solutions are as follows:

According to a first aspect of an embodiment of the present disclosure, an image classification method is provided, the method comprising:

Process the target image that needs to be classified through the pre-trained CNN model to determine the features of the last pooling layer output included in the CNN model;

The determined features are used as the features of the target image, and the features of the target image are processed by a pre-trained RNN (Recurrent Neural Network, cyclic neural network) model to obtain multiple labels of the target image, and the multiple Each of the tags is used to indicate the category to which the image content in the target image belongs.

Optionally, the pre-trained RNN model is used to process the features of the target image to obtain multiple labels of the target image, including:

Using the features of the target image as the input of the RNN model, and processing the features of the target image through the RNN model to obtain a first label, the first label is one of the multiple labels ;

The first label is used as the input of the RNN model, and the features of the target image and the first label are processed by the RNN model to obtain a second label, and the second label is the multiple one of the tags;

Determine the number of cycles, the number of cycles refers to the number of times the RNN model cycles to process the input label and the features of the target image;

When the number of cycles is less than or equal to the preset number of times, continue to use the second label as the input of the RNN model, and perform the above operation in a loop until the number of cycles is greater than the preset number of times, and obtain the multiple tags.

Optionally, the pre-trained RNN model is used to process the features of the target image to obtain multiple labels of the target image, including:

Using the features of the target image as the input of the RNN model, and processing the features of the target image through the RNN model to obtain a first label, the first label is one of the multiple labels ;

The first label is used as the input of the RNN model, and the features of the target image and the first label are processed by the RNN model to obtain a second label, and the second label is the multiple one of the tags;

Determining the probability that the target image simultaneously belongs to the category corresponding to all the currently determined labels;

When the probability is greater than or equal to the preset probability, continue to use the second label as the input of the RNN model, and perform the above operations in a loop until the probability is less than the preset probability, and obtain the plurality of labels .

Optionally, before the pre-trained CNN model is used to process the target image, it also includes:

Determine the training sample set, the training sample set includes a plurality of images and a plurality of labels pre-set for each image;

According to the multiple images, the initialization CNN model is trained to obtain the trained CNN model and the features of each image in the multiple images;

According to the feature of each image in the plurality of images, the initialization RNN model is trained to obtain the trained RNN model and a plurality of labels for each image in the plurality of images;

When the error between the plurality of labels of each image in the plurality of images obtained through training and the plurality of labels set for the corresponding image in advance is greater than the preset error, adjust the initialization CNN model and the set by gradient descent method Describe the initialization RNN model, and return to the step of training the initialization CNN model according to the multiple images, until the multiple labels of each image in the multiple images obtained through training and the multiple labels set in advance for the corresponding image Until the error between labels is less than or equal to the preset error, the final trained CNN model is determined as the pre-trained CNN model, and the final trained RNN model is determined as the pre-trained RNN model.

According to a second aspect of an embodiment of the present disclosure, an image classification device is provided, the device comprising:

The first processing module is used to process the target image that needs to be classified through the pre-trained CNN model, so as to determine the characteristics of the output of the last pooling layer included in the CNN model;

The second processing module is used to use the determined features as the features of the target image, process the features of the target image through the pre-trained RNN model, and obtain multiple labels of the target image, and the multiple labels Each label in is used to indicate the category to which the image content in the target image belongs.

Optionally, the second processing module includes:

The first processing submodule is configured to use the features of the target image as the input of the RNN model, and process the features of the target image through the RNN model to obtain a first label, and the first label is one of the plurality of tags;

The second processing submodule is used to use the first label as the input of the RNN model, and process the features of the target image and the first label through the RNN model to obtain a second label, so The second tag is one of the plurality of tags;

The first determination submodule is used to determine the number of cycles, the number of cycles refers to the number of times the RNN model cycles to process the input label and the features of the target image;

The first loop submodule is configured to continue to use the second label as the input of the RNN model when the number of loops is less than or equal to a preset number of times, and perform the above operation in a loop until the number of loops is greater than the preset number of times. When the number of times is set, the multiple labels are obtained.

Optionally, the second processing module includes:

The first processing submodule is configured to use the features of the target image as the input of the RNN model, and process the features of the target image through the RNN model to obtain a first label, and the first label is one of the plurality of tags;

The second processing submodule is used to use the first label as the input of the RNN model, and process the features of the target image and the first label through the RNN model to obtain a second label, so The second tag is one of the plurality of tags;

The second determining submodule is used to determine the probability that the target image belongs to the category corresponding to all the currently determined labels;

The second loop submodule is configured to continue to use the second label as the input of the RNN model when the probability is greater than or equal to the preset probability, and perform the above operations in a loop until the probability is less than the preset probability , get the multiple labels.

Optionally, the device also includes:

The first determination module is used to determine the training sample set, the training sample set includes a plurality of images and a plurality of labels pre-set for each image;

The first training module is used to train the initialized CNN model according to the plurality of images, and obtain the trained CNN model and the features of each image in the plurality of images;

The second training module is used to train the initialized RNN model according to the characteristics of each image in the plurality of images, and obtain the trained RNN model and a plurality of labels for each image in the plurality of images;

The second determination module is used to adjust by the gradient descent method when the error between the multiple labels of each image in the multiple images obtained through training and the multiple labels pre-set for the corresponding image is greater than the preset error. The initializing CNN model and the initializing RNN model, and returning to the step of training the initializing CNN model according to the plurality of images, until the plurality of labels of each image in the plurality of images obtained through training and the prior For the error between a plurality of labels set corresponding to the image until the error is less than or equal to the preset error, and the final trained CNN model is determined as the pre-trained CNN model, and the final trained RNN model is determined as the Describe the pre-trained RNN model.

According to a third aspect of the embodiments of the present disclosure, another image classification device is provided, and the device includes:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to execute the image classification method described in the first aspect above.

According to the fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, the computer-readable storage medium stores instructions, and when the instructions are executed by a processor, the image classification method described in the above-mentioned first aspect is implemented .

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

In the embodiment of the present disclosure, the CNN model is used to determine the features of the target image, and the RNN model is used to process the features of the target image to obtain multiple labels of the target image. Since the features of the target image are output by the last pooling layer included in the CNN model, that is, after the CNN model obtains the features of the target image, it does not determine the probability of each category that the target image belongs to according to the features of the target image, and The method is to directly process the features of the target image through the RNN model to obtain multiple labels of the target image, so as to classify images including at least two types of content.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.

FIG. 1A is a schematic diagram of the architecture of a CNN model provided by an embodiment of the present disclosure.

FIG. 1B is a schematic diagram of pixel point distribution of an image provided by an embodiment of the present disclosure.

FIG. 1C is a schematic diagram of a pooling process provided by an embodiment of the present disclosure.

FIG. 1D is a schematic diagram of an architecture of an RNN model provided by an embodiment of the present disclosure.

Fig. 2 is a flowchart of an image classification method provided by an embodiment of the present disclosure.

Fig. 3 is a flowchart of another image classification method provided by an embodiment of the present disclosure.

Fig. 4 is a flowchart of another image classification method provided by an embodiment of the present disclosure.

Fig. 5A is a block diagram of an image classification device provided by an embodiment of the present disclosure.

Fig. 5B is a block diagram of another image classification device provided by an embodiment of the present disclosure.

Fig. 6 is a block diagram of another image classification device provided by an embodiment of the present disclosure.

Fig. 7 is a block diagram of another image classification device provided by an embodiment of the present disclosure.

detailed description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the invention as recited in the appended claims.

Before explaining the embodiment of the present disclosure in detail, the two neural network models involved in the embodiment of the present disclosure will be introduced first.

(1) CNN model

The CNN model refers to a neural network for image classification and recognition developed on the basis of the traditional multi-layer neural network. Compared with the traditional multi-layer neural network, the CNN model introduces convolutional algorithms and pooling algorithm. Among them, the convolution algorithm refers to a mathematical algorithm that performs weighted superposition of data in a local area, and the pooling algorithm refers to a mathematical algorithm that performs sampling processing on data in a local area.

The function of the CNN model will be described in detail below in conjunction with the architecture of the CNN model.

FIG. 1A is a schematic diagram of the architecture of a CNN model 101 provided by an embodiment of the present disclosure. As shown in FIG. 1A , the CNN model 101 includes an input layer 1011, a convolutional layer 1012, a convolutional layer 1014, a pooling layer 1013, and a pooling layer. layer 1015, and fully connected layer 1016. The input layer 1011 is connected to the convolutional layer 1012, the convolutional layer 1012 is connected to the pooling layer 1013, the pooling layer 1013 is connected to the convolutional layer 1014, the convolutional layer 1014 is connected to the pooling layer 1015, and the pooling layer 1015 is connected to the full connection Layers 1016 are connected.

Wherein, the input layer 1011 is used to determine the pixel values of all the pixels included in the input image, and transmit the pixel values of all the pixels of the image to the convolution layer 1012 . The convolutional layer 1012 performs the first convolution algorithm processing according to the received pixel values of all pixels, obtains the pixels after the first layer of convolution processing, and transmits the pixels after the first convolution processing to the pool layer 1013. The pooling layer 1013 performs the first pooling algorithm processing according to the pixel values of the pixels after the first convolution processing, obtains the pixels after the first pooling processing, and converts the pixels after the first pooling processing Points are passed to the next convolutional layer 1014.

The convolution layer 1014 performs the second convolution algorithm processing on the pixels after the first pooling processing, obtains the pixels after the second convolution processing, and transmits the pixels after the second convolution processing to The next pooling layer 1015. The pooling layer 1015 performs the second pooling algorithm processing on the pixels after the second convolution processing, obtains the pixels after the second pooling processing, and transmits the pixels after the second pooling processing to Fully connected layer 1016. The fully connected layer 1016 determines the probability that the image belongs to each of the preset multiple categories according to the pixel values of the pixel points after the second pooling process, so as to obtain the label of the image.

It should be noted that the process of the first convolution algorithm is basically the same as that of the second convolution algorithm, and the process of the first pooling algorithm is basically the same as that of the second pooling algorithm. That is, the convolutional layer 1012 and the convolutional layer 1014 process the received pixels basically the same, and the pooling layer 1013 and the pooling layer 1015 process the received pixels basically the same. Therefore, the following takes the process of the first convolution algorithm processing and the first pooling algorithm processing process as examples for illustration.

Fig. 1B is a schematic diagram of the pixel point distribution of an image provided by an embodiment of the present disclosure. As shown in Fig. 1B , when the convolutional layer 1012 receives all the pixels included in the image, the first preset order and the first preset Set the local area to determine the pixels of the first local area, and perform weighted superposition of the pixel values of all the pixels included in the first local area according to the preset weight, and determine the obtained pixel value as the first convolution The pixel value of the first pixel after processing.

Afterwards, move the first preset local area according to the first preset order, and the moving distance is one column or one row of pixels to obtain the pixels of the second partial area, and convert all the pixels included in the second partial area The pixel values of are weighted and superimposed according to the preset weight, and the obtained pixel value is determined as the pixel value of the second pixel after the first convolution process. And so on, until the last local area includes the last batch of pixels of the image, and the pixel values of all the pixels included in the last local area are weighted and superimposed according to the preset weight, and the obtained pixel value is determined as the first The pixel value of the last pixel after the second convolution process. At this time, the convolution layer 1012 obtains the pixel values of all pixels after the first convolution processing, and transmits the pixel values of all pixel points after the first convolution processing to the pooling layer 1013 .

Wherein, the first preset order may be from left to right and from top to bottom, or may be from top to bottom and from left to right. The first preset local area may be an area of 5*5 pixels, or may be an area of 6*6 pixels.

In addition, the preset weight is a weight set in advance for each pixel in the preset area. For example, for the local area shown in Figure 1B, the preset weights of all pixels included in the local area are 1/16. At this time, for the pixels in the first local area, the first local area The pixel value of each included pixel is multiplied by the preset weight, and then the multiplied pixel values are added to obtain the pixel value of the first pixel after the first convolution process.

As shown in Figure 1C, when the pooling layer 1013 receives the pixel values of all pixels after the first convolution process, according to the second preset order and the second preset local area, determine Select the maximum pixel value among the pixel values of all the pixel points included in the first local area, and determine the maximum pixel value as the pixel value of the first pixel point after the first pooling process.

Afterwards, move the second preset local area according to the second preset order, the moving distance is preset pixels, and the number of the preset pixels is the same as the number of columns or rows of the second preset partial area , to get the pixels of the second local area. The maximum pixel value among the pixel values of all the pixel points included in the second local area is determined as the pixel value of the second pixel point after the first pooling process. And so on, until the last local area includes the last batch of pixels among all the pixels after the first convolution process, and the maximum pixel value among the pixel values of all the pixels included in the last local area Determined as the pixel value of the last pixel after the first pooling process. At this time, the pooling layer 1013 obtains the pixel values of all pixels after the first pooling process, and transmits the pixel values of all pixels after the first pooling process to the convolution layer 1014 .

Wherein, the second preset order may be from left to right and from top to bottom, or may be from top to bottom and from left to right. The second preset local area may be an area of 2×2 pixels, or may be an area of 3×3 pixels.

For example, both the first preset order and the second preset order are from left to right and from top to bottom, the first preset area is an area of 5×5 pixels, and the second preset area is 2×2 pixels. If the image obtained by the input layer is an image of 28×28 pixels, at this time, the convolution layer 1012 obtains the pixels after the first convolution process as 24×24 pixels, and the first convolution layer obtained by the pooling layer 1013 The pixels after the secondary pooling process are 12×12 pixels.

In addition, it should be noted that the pooling layer can determine the maximum pixel value in the selected local area as the pixel value of the corresponding pixel after pooling processing. Of course, the pixel value of the pixel point after pooling processing can be determined in other ways. Pixel values. For example, the pooling layer may determine an average value of pixel values of all pixel points in the selected local area as the pixel value of the corresponding pixel point after pooling processing.

It is worth noting that the CNN model provided by the embodiment of the present disclosure may include multiple convolutional layers and multiple pooling layers, which are not specifically limited in this embodiment of the present disclosure. In FIG. 1A, only two convolutional layers and Two pooling layers are used as an example for illustration.

In addition, it should be noted that since the pixel value of a pixel is usually represented by three components of RGB (RED/GREEN/BULE, three primary color light modes), therefore, in the process of processing the pixel value of a pixel, it is necessary to separately The three components of RGB are processed separately. Therefore, in the CNN model 101 shown in FIG. The RGB three components of the point are processed.

(2) RNN model

Since the traditional neural network has no memory function, that is, for the traditional neural network, its input is independent data without context association. But in practical applications, the input is usually some serialized input with obvious context characteristics. For example, it is necessary to predict the playback content of the next frame in the video. At this time, the output of the neural network must depend on the previous input. That is, it is required that the neural network should have a memory function, and the RNN model is a neural network with a memory function.

Figure 1D is a schematic diagram of the architecture of a RNN model 102 provided by an embodiment of the present disclosure. As shown on the left side of Figure 1D, the RNN model includes a three-layer structure of an input layer 1021, a hidden layer 1022, and an output layer 1023, and the hidden layer 1022 is a ring structure. Wherein, the input layer 1021 is connected to the hidden layer 1022 , and the hidden layer 1022 is connected to the output layer 1023 .

In order to illustrate the function of the RNN model 102, the structure of the RNN model 102 shown in FIG. 1D is expanded according to the structure shown on the right side of FIG. 1D . Since the data received by the input layer of the RNN model is sorted according to a certain time sequence, that is, the data received by the input layer is sequence data, for the convenience of explanation, the sequence data is marked as x ₁ , x ₂ , ..., x _n , the times corresponding to each data in the sequence data are t ₁ , t ₂ , . . . , t _n .

As shown on the right side of Figure 1D, in the expanded RNN model, the data received by the input layer 1021 at time t1 is x ₁ , and the data x ₁ is transmitted to the hidden layer 1022, and the hidden layer 1022 processes _{x 1} process, and transmit the processed data to the output layer 1023 to obtain the output data at time _t1 . The data received by the input layer 1021 at time _t2 is _x2 , and the data _x2 is transmitted to the hidden layer 1022. At this time, the hidden layer 1022 processes the _x2 according to the output data at time _t1 , and the processed data x2 The data is transmitted to the output layer 1023 to obtain the output data at time _t2 . That is, at any time t _i , the hidden layer 1022 not only receives the data x _i transmitted by the input layer 1021 at time t _i , but also receives the output data at time t _i-1 , and according to the output data at time t _i-1 Data x _i is processed to obtain the output data at time t _i .

Application scenarios of the embodiments of the present disclosure are introduced below . In real life, when a user sees an image, the user can directly determine the category of the image based on the results seen by the eyes, such as seeing a cat image, determine the category of the image as a cat; seeing a dog , determine the category of the image as a dog. But for a computer, when the computer receives an image, it actually receives the pixel values of all the pixels included in the image. Therefore, the computer needs to determine the category of the image according to the received pixel values of all the pixels included in the image, that is, the computer needs to classify the image. The image classification method provided by the embodiments of the present disclosure is applied to a scenario where a computer classifies the image. For the convenience of subsequent description, the image that needs to be classified is called the target image.

In the related art, the computer classifies images mainly through a single-label classification method. When an image includes content of at least two categories, the single-label classification method can only determine that the category of the image is one of the at least two categories, but cannot determine that the image belongs to the at least two categories at the same time, That is, the above-mentioned single-label classification method cannot classify images including at least two types of content.

Therefore, in the embodiment of the present disclosure, after the features of the target image are determined by the CNN model, the features of the target image are processed by the RNN model to obtain multiple labels of the target image. Since the features of the target image are output by the last pooling layer included in the CNN model, that is, after the CNN model obtains the features of the target image, it does not determine the probability of each category that the target image belongs to according to the features of the target image, and The method is to directly process the features of the target image through the RNN model to obtain multiple labels of the target image, so as to classify images including at least two types of content.

The image classification method provided by the embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

FIG. 2 is a flow chart of an image classification method provided by an embodiment of the present disclosure. The method is applied to any device that needs to classify images, and the device may be a terminal or a server. As shown in Figure 2, the method includes the following steps.

In step 201, the pre-trained CNN model is used to process the target image to be classified, so as to determine the features output by the last pooling layer included in the CNN model.

In step 202, the determined features are used as the features of the target image, and the features of the target image are processed by the pre-trained RNN model to obtain multiple labels of the target image, and each label in the multiple labels is used to indicate the category to which the image content in the target image belongs.

In the embodiment of the present disclosure, the CNN model is used to determine the features of the target image, and the RNN model is used to process the features of the target image to obtain multiple labels of the target image. Since the features of the target image are output by the last pooling layer included in the CNN model, that is, after the CNN model obtains the features of the target image, it does not determine the probability of each category that the target image belongs to according to the features of the target image, and The method is to directly process the features of the target image through the RNN model to obtain multiple labels of the target image, so as to classify images including at least two types of content.

Optionally, the features of the target image are processed by the pre-trained RNN model to obtain multiple labels of the target image, including:

The feature of the target image is used as the input of the RNN model, and the feature of the target image is processed by the RNN model to obtain a first label, and the first label is one of the multiple labels;

Using the first label as the input of the RNN model, and processing the features of the target image and the first label through the RNN model to obtain a second label, which is one of a plurality of labels;

Determine the number of cycles, the number of cycles refers to the number of times the RNN model cycles to process the input label and the features of the target image;

When the number of cycles is less than or equal to the preset number of times, the second label continues to be used as the input of the RNN model, and the above operations are performed cyclically until the number of cycles is greater than the preset number of times, and multiple labels are obtained.

Optionally, the features of the target image are processed by the pre-trained RNN model to obtain multiple labels of the target image, including:

The feature of the target image is used as the input of the RNN model, and the feature of the target image is processed by the RNN model to obtain a first label, and the first label is one of the multiple labels;

The first label is used as the input of the RNN model, and the feature of the target image and the first label are processed by the RNN model to obtain a second label, and the second label is one of the plurality of labels;

Determine the probability that the target image belongs to the categories corresponding to all the currently determined labels;

When the probability is greater than or equal to the preset probability, the second label is continuously used as the input of the RNN model, and the above operations are performed cyclically until the probability is less than the preset probability, and the multiple labels are obtained.

Optionally, before processing the target image through the pre-trained CNN model, it also includes:

Determine a training sample set, the training sample set includes a plurality of images and a plurality of labels pre-set for each image;

According to the multiple images, the initialization CNN model is trained to obtain the trained CNN model and the features of each image in the multiple images;

According to the feature of each image in the plurality of images, the initialization RNN model is trained to obtain the trained RNN model and multiple labels of each image in the plurality of images;

When the error between the multiple labels of each image in the multiple images obtained through training and the multiple labels set for the corresponding image in advance is greater than the preset error, the initialization CNN model and the initialization RNN are adjusted by the gradient descent method model, and return to the step of training the initialized CNN model according to the multiple images until the error between the multiple labels of each image in the multiple images obtained through training and the multiple labels pre-set for the corresponding image is less than or equal to the preset error, and the final trained CNN model is determined as the pre-trained CNN model, and the final trained RNN model is determined as the pre-trained RNN model.

All the above optional technical solutions may be combined in any way to form optional embodiments of the present disclosure, which will not be described in detail in the embodiments of the present disclosure.

FIG. 3 is a flow chart of another image classification method provided by an embodiment of the present disclosure. The method is applied to any device that needs to classify images, and the device may be a terminal or a server. As shown in Figure 3, the method includes the following steps.

In step 301, a pre-trained CNN model and a pre-trained RNN model are determined.

Since the embodiment of the present disclosure classifies the target image through the CNN model and the RNN model, before classifying the target image, the CNN model and the RNN model need to be trained first, that is, the pre-trained CNN model and the RNN model must be determined. Pre-trained RNN models.

Wherein, the implementation process of step 301 may be: determine a training sample set, the training sample set includes a plurality of images and a plurality of labels pre-set for each image. According to the plurality of images, the initialized CNN model is trained to obtain the trained CNN model and the features of each image in the plurality of images. According to the feature of each image in the plurality of images, the initialized RNN model is trained to obtain the trained RNN model and multiple labels of each image in the plurality of images.

When the error between the multiple labels of each image in the multiple images obtained through training and the multiple labels set for the corresponding image in advance is greater than the preset error, the initialization CNN model and the initialization RNN model are adjusted by the gradient descent method , and return to the step of training the initialized CNN model according to the multiple images until the error between the multiple labels of each image in the multiple images obtained through training and the multiple labels pre-set for the corresponding image is less than or equal to the preset error, and the final trained CNN model is determined as the pre-trained CNN model, and the final trained RNN model is determined as the pre-trained RNN model.

That is, after the first training of the initialized CNN model and the initialized RNN model, according to the plurality of labels of each image in the plurality of images obtained through training and the plurality of labels set in advance for the corresponding images, it is judged whether it is necessary to continue Continue training on the initialized CNN model and initialized RNN. If it is determined that it is necessary to continue training the initialized CNN model and initialized RNN, adjust the initialized CNN model and initialized RNN according to the above method to obtain the first adjusted CNN model and RNN model, and initialize the CNN model and RNN according to the above training. The process of initializing the RNN model, training the CNN model adjusted for the first time and the RNN model adjusted for the first time, and obtaining the multiple labels of each image in the multiple images obtained in this training, and returning to execute according to the training The multiple labels of each image in the multiple images and the multiple labels set in advance for the corresponding images, determine whether to continue the operation of training the initialized CNN model and initialized RNN, and execute the above process cyclically. Until it is judged that the adjusted CNN model and the adjusted RNN do not need to be continuously trained.

Wherein, the training sample set is a predetermined set of images, and the set of images includes multiple images and multiple labels pre-set for each image. Setting multiple tags for each image can be achieved by adding the manager, that is, when receiving the tag adding instruction sent by the manager through the preset operation, the multiple tags carried in the tag adding instruction are set as the Multiple labels per image.

For example, when the training sample set includes 1000 images, the manager can pre-add 5 labels to each of the 1000 images, that is, the training sample set includes 1000 images and 5 labels pre-set for each image Label.

In addition, the initialized CNN model and the initialized RNN model are adjusted by the gradient descent method, that is, according to the errors between the multiple labels of each image in the multiple images obtained through training and the multiple labels set in advance for the corresponding images, A gradient vector is determined, and the gradient vector is a direction vector in which errors between multiple labels of each image in the multiple images obtained through training and multiple labels pre-set for the corresponding images decrease. According to the gradient vector, the initialization CNN model and the RNN model are adjusted.

Since the gradient vector is the direction vector of the error drop between the multiple labels of each image in the multiple images obtained through training and the multiple labels set for the corresponding image in advance, when the adjusted CNN model and the adjusted RNN When the model re-trains the training sample set, compared to the error between the multiple labels of each image in the multiple images obtained in the previous training and the multiple labels set for the corresponding images in advance, the multiple images obtained in this training The error between the multiple labels for each image in the image and the multiple labels pre-set for the corresponding image will drop.

In step 302, the pre-trained CNN model is used to process the target image to be classified, so as to determine the features output by the last pooling layer included in the CNN model.

Step 302 is described in detail by taking the CNN model shown in FIG. 1A as an example. That is, the pre-trained CNN model includes two pairs of convolution-pooling layers, each pair of convolution-pooling layers includes a convolutional layer and a pooling layer.

As shown in FIG. 1A , the input layer 1011 included in the pre-trained CNN model transmits the pixel values of all pixels included in the target image to the first pair of convolution-pooling layers included in the CNN model. The first pair of convolution-pooling layers processes the target image according to the pixel values of all pixels included in the target image, and transmits the processed pixels to the next pair of convolution-pooling layers. The next pair of convolution-pooling layers continues to process the processed pixels until the second pair of convolution-pooling layers outputs the features of the target image.

Wherein, the first convolution layer 1012 included in the first pair of convolution-pooling layers performs the first convolution algorithm processing on all the pixels included in the target image to obtain the pixels after the first convolution processing, and The pixels after the first convolution process are input to the first pooling layer 1013 included in the first pair of convolution-pooling layers. The first pooling layer 1013 performs the first pooling algorithm processing on the pixels after the first convolution processing, obtains the pixels after the first pooling processing, and converts the pixels after the first pooling processing It is transmitted to the second pair of convolution-pooling layers, and the second pair of convolution-pooling layers performs the second convolution algorithm processing and the second pooling algorithm processing in the same way as above. .

Among them, the detailed process of the first convolution algorithm processing and the second convolution algorithm processing, as well as the first pooling algorithm processing and the second pooling algorithm processing can refer to the convolution algorithm of the CNN model shown in Figure 1A The processing process and pooling algorithm processing process will not be described in detail here.

It should be noted that, compared with the process of classifying images by the CNN model shown in FIG. 1A , when the pre-trained CNN model provided by the embodiment of the present disclosure determines the features output by the last pooling layer included in the CNN model, , the features output by the last pooling layer are not transferred to the fully connected layer, but the features output by the last pooling layer are transferred to the pre-trained RNN model.

That is, when the features output by the last pooling layer included in the CNN model are determined, the determined features are used as the features of the target image, and the features of the target image are processed by the pre-trained RNN model to obtain the Multiple tags of the target image, each of the multiple tags is used to indicate the category to which the image content in the target image belongs. Wherein, the features of the target image are processed by the pre-trained RNN model, and multiple labels of the target image can be obtained through the following steps 303 to 306 .

In step 303, the feature of the target image is used as an input of the RNN model, and the feature of the target image is processed by the RNN model to obtain a first label, and the first label is one of the multiple labels.

It can be seen from step 301 that the pre-trained RNN model is determined according to multiple images in the training sample set and multiple labels pre-set for each image. Therefore, when the features of the target image are used as the input of the RNN model , the RNN model can output a label, that is, obtain the first label among the multiple labels.

Taking the RNN model shown in Figure 1D as an example for illustration, when the input layer of the RNN model receives the features of the target image at time _t1 , the input layer transmits the features of the target image to the hidden layer, and the hidden layer takes the target image The features are processed to get the first label. That is, the first label is the output data at time _t1 , the hidden layer transmits the first label to the output layer, and the output layer outputs the first label.

In step 304, the first label is used as an input of the RNN model, and the features of the target image and the first label are processed through the RNN model to obtain a second label, which is one of multiple labels.

As shown in Figure 1D, since the input layer of the RNN model did not receive the next data at time _t2 , the hidden layer continues to determine the feature of the target image received by the input layer of the RNN model at time _t1 as the time _t2 Input data x ₂ , and continue to process the features of the target image according to the first label of the output data at time t ₁ to obtain the second label.

That is, at any time t _i shown in Figure 1D, the hidden layer of the RNN model directly determines the features of the target image received by the input layer of the RNN model at time t ₁ as the input data _{xi at time t i ,} and according to The i-1th label obtained at the previous moment continues to process the characteristics of the target object to obtain the i-th label.

In step 305, the number of cycles is determined, and the number of cycles refers to the number of times the RNN model cycles to process the input labels and the features of the target image.

In the RNN model shown in FIG. 1D , since the RNN model is a cyclic neural network model, in order to prevent the RNN model from infinitely looping in practical applications, it is usually necessary to set a loop termination condition for the RNN model.

In a possible implementation manner, a termination condition may be set for the RNN model through the number of cycles. That is, when the RNN model performs the cycle as shown in FIG. 1D , after obtaining a label, the number of cycles is determined. And according to the number of cycles, it is judged whether the cycle operation needs to be continued. Wherein, the process of judging whether the loop operation needs to be continued can be realized through the following step 306 .

In step 306, when the number of cycles is less than or equal to the preset number of times, the second label is continuously used as the input of the RNN model, and the above operations are performed cyclically until the number of cycles is greater than the preset number of times, and multiple labels are obtained.

Wherein, after the second label is obtained, the number of cycles is determined to be 1; after the third label is obtained, the number of cycles is determined to be 2; and so on, after the i-th label is obtained, the number of cycles is determined to be i-1.

The preset number of times is a preset number of times, and the preset number of times can be 5, 10 or 15, etc. For example, the preset number of times is 10. After getting the i-th label, determine the number of cycles to be i-1, and judge whether the number of cycles i-1 is greater than 5. When i-1 is less than or equal to greater than 5, continue the above cycle Operate until it is judged that i-1 is greater than 5, at this time, multiple labels of the target image are obtained.

In the embodiment of the present disclosure, the CNN model is used to determine the features of the target image, and the RNN model is used to process the features of the target image to obtain multiple labels of the target image. Since the features of the target image are output by the last pooling layer included in the CNN model, that is, after the CNN model obtains the features of the target image, it does not determine the probability of each category that the target image belongs to according to the features of the target image, and The method is to directly process the features of the target image through the RNN model to obtain multiple labels of the target image, so as to classify images including at least two types of content.

Optionally, during the cycle process of the RNN model as shown in Figure 1D, in addition to setting the loop termination condition for the RNN model through steps 305 and 306 in the embodiment of Figure 3, it is also possible to set the cycle termination condition for the RNN model in other ways To set the loop termination condition, the following embodiment will provide another implementation manner of setting the loop termination condition.

FIG. 4 is a flow chart of another image classification method provided by an embodiment of the present disclosure. The method is applied to any device that needs to classify images, and the device may be a terminal or a server. As shown in Figure 4, the method includes the following steps.

In step 401, a pre-trained CNN model and a pre-trained RNN model are determined.

For the implementation process of step 401, reference may be made to the implementation process of step 301 in the embodiment of FIG. 3 , which will not be described in detail here.

In step 402, the pre-trained CNN model is used to process the target image to be classified, so as to determine the features output by the last pooling layer included in the CNN model.

For the implementation process of step 402, reference may be made to the implementation process of step 302 in the embodiment of FIG. 3 , which will not be described in detail here.

At this time, the determined features are used as the features of the target image, and the features of the target image are processed through the pre-trained RNN model to obtain multiple labels of the target image, and each label in the multiple labels is used for Indicates the category to which the image content in this target image belongs. Wherein, the features of the target image are processed by the pre-trained RNN model to obtain multiple labels of the target image through the following steps 403 to 406 .

In step 403, the feature of the target image is used as an input of the RNN model, and the feature of the target image is processed by the RNN model to obtain a first label, and the first label is one of the multiple labels.

For the implementation process of step 403, reference may be made to the implementation process of step 303 in the embodiment of FIG. 3 , which will not be described in detail here.

In step 404, the first label is used as the input of the RNN model, and the features of the target image and the first label are processed through the RNN model to obtain a second label, which is one of the multiple labels .

For the implementation process of step 404, reference may be made to the implementation process of step 304 in the embodiment of FIG. 3 , which will not be described in detail here.

In step 405, the probability that the target image belongs to categories corresponding to all currently determined tags is determined.

The embodiment of the present disclosure provides another implementation manner of setting a loop termination condition for the RNN model. That is, in the cyclic process shown in FIG. 1D by the RNN model, after obtaining a label, determine the probability that the target image belongs to the categories corresponding to all the currently determined labels. And according to the probability, it is judged whether to continue the loop operation. Wherein, the process of judging whether the loop operation needs to be continued can be realized through the following step 406 .

Because in the existing RNN model, at any time t _i , the RNN model will determine the probability of simultaneous occurrence of all output data currently obtained after determining the output data at time t _i . Therefore, in the embodiment of the present invention, after each label is obtained, the RNN model will determine the probability that all the currently obtained labels appear at the same time, that is, determine that the target image belongs to the category corresponding to all the currently determined labels at the same time The probability.

As the number of cycles increases, the probability that all output data currently obtained will appear at the same time will become smaller and smaller, that is, as the number of cycles increases, the probability that the target image belongs to the category corresponding to all the currently determined labels will also get smaller. Therefore, a preset probability may be set and determined as a loop termination condition. That is, during the cycle of the RNN model, after a label is obtained, the probability is determined. And according to the probability, it is judged whether to continue the loop operation. Wherein, the process of judging whether the loop operation needs to be continued can be realized through the following step 306 .

In step 406, when the probability is greater than or equal to the preset probability, the second label is continuously used as the input of the RNN model, and the above operations are performed cyclically until the probability is less than the preset probability, and the multiple labels are obtained.

Wherein, the preset probability is a preset probability, and the preset probability may be 0.01, 0.02, or 0.03. For example, the preset probability is 0.01. After obtaining the i-th label, determine the probability that the target image belongs to the category corresponding to all the currently determined labels at the same time, and judge whether the probability is greater than or equal to 0.01. When the probability is greater than or equal to If it is greater than 0.01, continue the above loop operation until it is judged that the probability is less than 0.01, at this time, multiple labels of the target image are obtained.

It should be noted that, in the embodiment of the present disclosure, the loop termination condition can be set for the RNN model through steps 305 and 306 in the embodiment of FIG. The RNN model sets the loop termination condition. Of course, in practical applications, it is also possible to set the loop termination condition for the RNN model through steps 305 and 306 in the embodiment of FIG. 3, and steps 405 and 406 in the embodiment of FIG. Setup has two loop termination conditions. At this time, in the cyclic process of the RNN model as shown in Figure 1D, after obtaining a label, determine the number of cycles and the above-mentioned probability at the same time, when one of the number of cycles and the above-mentioned probability satisfies the corresponding cycle termination condition, you can Terminate the looping process of the RNN model.

In the embodiment of the present disclosure, the CNN model is used to determine the features of the target image, and the RNN model is used to process the features of the target image to obtain multiple labels of the target image. Since the features of the target image are output by the last pooling layer included in the CNN model, that is, after the CNN model obtains the features of the target image, it does not determine the probability of each category that the target image belongs to according to the features of the target image, and The method is to directly process the features of the target image through the RNN model to obtain multiple labels of the target image, so as to classify images including at least two types of content.

In addition to providing the above image classification method, the embodiment of the present disclosure also provides an image classification device, and the image classification device provided by the embodiment of the present disclosure will be described in detail below.

FIG. 5A is a block diagram of an image classification apparatus 500 provided by an embodiment of the present disclosure. As shown in FIG. 5A , the image classification apparatus 500 includes a first processing module 501 and a second processing module 502 .

The first processing module 501 is used to process the target image that needs to be classified through the pre-trained CNN model, so as to determine the features output by the last pooling layer included in the CNN model;

The second processing module 502 is used to use the determined features as the features of the target image, process the features of the target image through the pre-trained RNN model, and obtain multiple labels of the target image, each of the multiple labels A tag is used to indicate the category to which the image content in the target image belongs.

Optionally, the second processing module includes:

The first processing submodule is used to use the feature of the target image as the input of the RNN model, and process the feature of the target image through the RNN model to obtain a first label, and the first label is one of the multiple labels one;

The second processing submodule is used to use the first label as the input of the RNN model, and process the features of the target image and the first label through the RNN model to obtain a second label, and the second label is the plurality of labels one of the;

The first determination submodule is used to determine the number of cycles, which refers to the number of times the RNN model cycles to process the input label and the features of the target image;

The first loop sub-module is used to continue to use the second label as the input of the RNN model when the number of loops is less than or equal to the preset number of times, and perform the above operations in a loop until the number of loops is greater than the preset number of times to obtain the multiple tags.

Optionally, the second processing module 502 includes:

The first processing submodule is used to use the feature of the target image as the input of the RNN model, and process the feature of the target image through the RNN model to obtain a first label, and the first label is one of the multiple labels one;

The second processing submodule is used to use the first label as the input of the RNN model, and process the features of the target image and the first label through the RNN model to obtain a second label, and the second label is the plurality of labels one of the;

The second determining submodule is used to determine the probability that the target image belongs to the category corresponding to all the currently determined labels;

The second loop sub-module is used to continue to use the second label as the input of the RNN model when the probability is greater than or equal to the preset probability, and perform the above operations in a loop until the probability is less than the preset probability to obtain the plurality of labels .

Optionally, referring to FIG. 5B, the apparatus 500 further includes a first determination module 503, a first training module 504, a second training module 505, and a second determination module 506:

The first determination module 503 is used to determine a training sample set, the training sample set includes a plurality of images and a plurality of labels pre-set for each image;

The first training module 504 is used to train the initialized CNN model according to the multiple images, and obtain the trained CNN model and the features of each image in the multiple images;

The second training module 505 is used to train the initialized RNN model according to the characteristics of each image in the plurality of images, and obtain the trained RNN model and a plurality of labels for each image in the plurality of images;

The second determination module 506 is used to adjust the values by gradient descent method when the error between the multiple labels of each image obtained through training and the multiple labels pre-set for the corresponding image is greater than the preset error. The initialized CNN model and the initialized RNN model, and return to the step of training the initialized CNN model according to the plurality of images, until the plurality of labels of each image in the plurality of images obtained through training and the corresponding image settings in advance Until the errors between multiple labels are less than or equal to the preset error, the final trained CNN model is determined as the pre-trained CNN model, and the final trained RNN model is determined as the pre-trained RNN model.

In the embodiment of the present disclosure, the CNN model is used to determine the features of the target image, and the RNN model is used to process the features of the target image to obtain multiple labels of the target image. Since the features of the target image are output by the last pooling layer included in the CNN model, that is, after the CNN model obtains the features of the target image, it does not determine the probability of each category that the target image belongs to according to the features of the target image, and The method is to directly process the features of the target image through the RNN model to obtain multiple labels of the target image, so as to classify images including at least two types of content.

Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

Fig. 6 is a block diagram of another image classification apparatus 600 provided by an embodiment of the present disclosure. For example, the apparatus 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

6, device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input/output (I/O) interface 612, sensor component 614, and communication component 616 .

The processing component 602 generally controls the overall operations of the device 600, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 602 may include one or more modules that facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602 .

The memory 604 is configured to store various types of data to support operations at the device 600 . Examples of such data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, and the like. The memory 604 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power component 606 provides power to various components of device 600 . Power components 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 600 .

The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 608 includes a front camera and/or a rear camera. When the device 600 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 610 is configured to output and/or input audio signals. For example, the audio component 610 includes a microphone (MIC) configured to receive external audio signals when the device 600 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 604 or sent via communication component 616 . In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

The I/O interface 612 provides an interface between the processing component 602 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 614 includes one or more sensors for providing status assessments of various aspects of device 600 . For example, the sensor component 614 can detect the open/closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor component 614 can also detect a change in the position of the device 600 or a component of the device 600 , the presence or absence of user contact with the device 600 , the device 600 orientation or acceleration/deceleration and the temperature change of the device 600 . The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 614 may also include optical sensors, such as CMOS or CCD image sensors, for use in imaging applications. In some embodiments, the sensor component 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 616 is configured to facilitate wired or wireless communication between the apparatus 600 and other devices. The device 600 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 600 may be programmed by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Realized by a gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, it is used to execute the image classification method provided by the embodiments shown in the above-mentioned Fig. 2 , Fig. 3 and Fig. 4 .

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as the memory 604 including instructions, which can be executed by the processor 620 of the device 600 to implement the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

A non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the terminal, the terminal can execute the image classification method provided by the embodiments shown in FIG. 2 , FIG. 3 and FIG. 4 .

Fig. 7 is a block diagram of another image classification apparatus 700 provided by an embodiment of the present disclosure. For example, the apparatus 700 may be provided as a server. 7, the apparatus 700 includes a processor 722, which further includes one or more processors, and a memory resource represented by a memory 732 for storing instructions executable by the processor 722, such as application programs. The application program stored in memory 732 may include one or more modules each corresponding to a set of instructions. In addition, the processor 722 is configured to execute instructions to execute the image classification method provided in the above embodiments shown in FIG. 2 , FIG. 3 and FIG. 4 .

Device 700 may also include a power component 726 configured to perform power management of device 700 , a wired or wireless network interface 750 configured to connect device 700 to a network, and an input-output (I/O) interface 758 . The apparatus 700 can operate based on an operating system stored in the memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as the memory 732 including instructions, which can be executed by the processor 722 of the device 700 to implement the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the server, the server can execute the image classification method provided by the above-mentioned embodiments shown in FIG. 2 , FIG. 3 and FIG. 4 .

Other embodiments of the invention will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present invention, these modifications, uses or adaptations follow the general principles of the present invention and include common knowledge or conventional technical means in the technical field not disclosed in this disclosure . The specification and examples are to be considered exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It should be understood that the present invention is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims (10)

1. a kind of image classification method, it is characterised in that methods described includes：

The target image that needs are classified is handled by the convolutional neural networks CNN models of training in advance, to determine The feature for last the pond layer output that the CNN models include；

Using the feature of determination as the target image feature, by the Recognition with Recurrent Neural Network RNN models of training in advance to institute The feature for stating target image is handled, and obtains each label in multiple labels of the target image, the multiple label For indicating the classification belonging to the picture material in the target image.

2. according to the method described in claim 1, it is characterised in that the RNN models by training in advance are to the target The feature of image is handled, and obtains multiple labels of the target image, including：

Using the feature of the target image as the RNN models input, and by the RNN models to the target image Feature handled, obtain the first label, first label is one in the multiple label；

Using first label as the input of the RNN models, and pass through feature of the RNN models to the target image Handled with first label, obtain the second label, second label is one in the multiple label；

Determine cycle-index, the cycle-index refers to the RNN mold cycles to the label of input and the target image Feature carries out number of processing；

When the cycle-index is less than or equal to preset times, second label is continued to be used as the defeated of the RNN models Enter, circulation performs aforesaid operations, until when the cycle-index is more than the preset times, obtaining the multiple label.

3. according to the method described in claim 1, it is characterised in that the RNN models by training in advance are to the target The feature of image is handled, and obtains multiple labels of the target image, including：

Using the feature of the target image as the RNN models input, and by the RNN models to the target image Feature handled, obtain the first label, first label is one in the multiple label；

Using first label as the input of the RNN models, and pass through feature of the RNN models to the target image Handled with first label, obtain the second label, second label is one in the multiple label；

The target image is determined while belonging to the probability for the corresponding classification of all labels for currently determining to obtain；

When the probability is more than or equal to predetermined probabilities, second label is continued into the input as the RNN models, followed Ring performs aforesaid operations, until when the probability is less than the predetermined probabilities, obtaining the multiple label.

4. according to any described method of claims 1 to 3, it is characterised in that the CNN models by training in advance are to mesh Before logo image is handled, in addition to：

Training sample set is determined, the training sample set includes multiple images and the multiple labels set in advance for every image；

According to multiple described images, initialization CNN models are trained, CNN models and multiple described figures after being trained The feature of every image as in；

According to the feature of every image in multiple described images, initialization RNN models are trained, the RNN after being trained Multiple labels of every image in model and multiple described images；

Described in obtained by training in multiple images multiple labels of every image with advance for correspondence image set it is many When error between individual label is more than default error, the initialization CNN models are adjusted and described initial by gradient descent method Change RNN models, and return according to multiple described images, the step of being trained to initialization CNN models, until by training Error in multiple images described between the multiple labels and the advance multiple labels set for correspondence image of every image Untill less than or equal to the default error, and by the CNN moulds for finally training obtained CNN models to be defined as the training in advance Type, by the RNN models for finally training obtained RNN models to be defined as the training in advance.

5. a kind of image classification device, it is characterised in that described device includes：

First processing module, the target figure that needs are classified for the convolutional neural networks CNN models by training in advance As being handled, to determine the feature for last the pond layer output that the CNN models include；

Second processing module, for as the feature of the target image, the feature of determination to be passed through into the circulation god of training in advance The feature of the target image is handled through network RNN models, multiple labels of the target image are obtained, it is the multiple Each label in label is used to indicate the classification belonging to the picture material in the target image.

6. device according to claim 5, it is characterised in that the Second processing module includes：

First processing submodule, for using the feature of the target image as the RNN models input, and by described RNN models are handled the feature of the target image, obtain the first label, first label is in the multiple label One；

Second processing submodule, for as the input of the RNN models, and first label to be passed through into the RNN models Feature and first label to the target image are handled, and obtain the second label, and second label is described many One in individual label；

First determination sub-module, for determining cycle-index, the cycle-index refers to mark of the RNN mold cycles to input The feature of label and the target image carries out number of processing；

First circulation submodule, for when the cycle-index is less than or equal to preset times, second label to be continued As the input of the RNN models, circulation performs aforesaid operations, until when the cycle-index is more than the preset times, obtaining To the multiple label.

7. device according to claim 5, it is characterised in that the Second processing module includes：

First processing submodule, for using the feature of the target image as the RNN models input, and by described RNN models are handled the feature of the target image, obtain the first label, first label is in the multiple label One；

Second processing submodule, for as the input of the RNN models, and first label to be passed through into the RNN models Feature and first label to the target image are handled, and obtain the second label, and second label is described many One in individual label；

Second determination sub-module, for determining the target image while belonging to the corresponding class of all labels for currently determining to obtain Other probability；

Second circulation submodule, for when the probability be more than or equal to predetermined probabilities when, will second label continuation as The input of the RNN models, circulation performs aforesaid operations, until when the probability is less than the predetermined probabilities, obtaining described many Individual label.

8. according to any described device of claim 5 to 7, it is characterised in that described device also includes：

First determining module, for determining training sample set, the training sample set includes multiple images and is in advance every figure As the multiple labels set；

First training module, for according to multiple described images, being trained to initialization CNN models, the CNN after being trained The feature of every image in model and multiple described images；

Second training module, for the feature according to every image in multiple described images, is instructed to initialization RNN models Practice, multiple labels of every image in the RNN models and multiple described images after being trained；

Second determining module, for multiple labels when every image in multiple images described in being obtained by training with being in advance When error between multiple labels that correspondence image is set is more than default error, the initialization is adjusted by gradient descent method CNN models and the initialization RNN models, and return according to multiple described images, the step being trained to initialization CNN models Suddenly, it is many of correspondence image setting up to multiple labels of every image in multiple images described in being obtained by training and in advance Untill error between individual label is less than or equal to the default error, and obtained CNN models will be finally trained to be defined as institute The CNN models of training in advance are stated, by the RNN models for finally training obtained RNN models to be defined as the training in advance.

9. a kind of image classification device, it is characterised in that described device includes：

Processor；

Memory for storing processor-executable instruction；

Wherein, the processor is configured as the step of perform claim requires any one method described in 1-4.

10. be stored with instruction on a kind of computer-readable recording medium, the computer-readable recording medium, it is characterised in that The step of instruction realizes any one method described in claim 1-4 when being executed by processor.