JP5499362B2 - Semi-teacher signal recognition search apparatus, semi-teacher signal recognition search method, and program - Google Patents

Description

  The present invention relates to a semi-teacher signal recognition search apparatus, a semi-teacher signal recognition search method, and a program.

  In recent years, an image retrieval technique for retrieving a desired image based on given language information, and an image recognition technique for automatically assigning language information that describes the image to a given image are digital cameras, With the spread of imaging devices such as mobile phones and the generalization of image sharing on the Internet, it has become a very important technology. In addition, many image recognition search technologies that realize image search and image recognition under the same framework have been developed (see, for example, Non-Patent Document 1).

Nakayama, Harada, Kuniyoshi, Otsu “Ultra-high-speed image recognition / retrieval method using probabilistic structure learning corresponding to image / word concept”, IEICE Technical Report, PRMU2007-147, December 2007

  However, in the technique described in Non-Patent Document 1, it is necessary to learn the relationship between an image and language information before actually performing search or recognition. Many pairs of images and language information are required. However, manually adding linguistic information to a large number of images involves a lot of labor, and it is difficult to summon a large number of images associated with linguistic information. If the image associated with the language information cannot be prepared sufficiently, the relationship between the image and the language information cannot be learned with high accuracy, and as a result, the reliability of image recognition and image search is impaired. There's a problem.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an image (signal) and language information (in the case where only a small amount of an image (signal) associated with language information (additional information) is available. A semi-teacher signal recognition / retrieval device, a semi-teacher signal recognition / retrieval method, and a program capable of accurately learning the relationship of (additional information) and improving the accuracy of image (signal) retrieval or image (signal) recognition. It is in.

The present invention has been made to solve the above problems, and one aspect of the present invention is an element of a completely accumulated signal set, which is a set of signals to which additional information, which is information for explaining a signal, is given in advance. Extraction of stored signal features, which are vectors representing the characteristics of each signal, from each of the complete stored signal and the incomplete stored signal that is an element of the incomplete stored signal set that is a set of signals to which no additional information is given. And a storage addition that is a vector representing the characteristics of the stored additional information from each of the stored additional information that is an element of the stored additional information set that is a set of additional information given to the complete stored signal. A storage additional information feature extraction unit that extracts information features, a complete storage signal feature set that is a set of storage signal features of the complete storage signal, and a set of storage signal features of the incomplete storage signal A storage latent variable that extracts a storage latent variable that is a variable for describing the relationship between a signal and additional information from a certain incompletely stored signal feature set and a stored additional information feature set that is a set of the stored additional information features From the extraction unit, the complete accumulated signal feature set, the incomplete accumulated signal feature set, the accumulated additional information feature set, and the accumulated latent variable set that is a set of the accumulated latent variables, a signal and additional information A topic model learning unit that extracts a topic model, which is a model describing the relationship, and input additional information that, when input additional information is input, extracts input additional information features that are vectors representing the characteristics of the input additional information A feature extraction unit, which gives the input additional information feature to the topic model and relates to the input additional information from the complete accumulation signal set and the incomplete accumulation signal set And a signal retrieval section for selecting items, the storage latent variable extraction unit is configured entirely of stored signal feature set and the incomplete stored signal feature set and the accumulation additional information feature set, multi-accumulation potential variable exists A latent variable space generation unit that generates a latent variable space that is a dimension vector space, the complete accumulated signal feature set, the accumulated additional information feature set, and the latent variable space are used to correspond to the complete accumulated signal feature set. An incomplete accumulation is performed by using a complete accumulation latent variable set extraction unit that extracts a complete accumulation latent variable set that is an accumulation latent variable set, the complete accumulation signal feature set, the incomplete accumulation signal feature set, and the latent variable space. An incompletely stored latent variable set that is a stored latent variable set corresponding to the signal feature set is extracted, and the incompletely stored latent variable set and the complete stored latent variable set are combined and stored. Is a semi-teacher signal recognition retrieval apparatus according to claim Rukoto a storage latent variable set extraction unit that forms the entire latent variable set.

Further, according to one embodiment of the present invention, a complete accumulation signal that is an element of a complete accumulation signal set that is a set of signals to which additional information that is information that describes a signal is given in advance, and a signal that is not given additional information. A storage signal feature extraction unit that extracts a storage signal feature that is a vector expressing the characteristics of each signal from each of the incomplete storage signals that are elements of the incomplete storage signal set that is a set, and is provided to the complete storage signal A storage additional information feature extraction unit that extracts a storage additional information feature that is a vector expressing the characteristics of the storage additional information from each of the storage additional information that is an element of the storage additional information set that is a set of additional information; A complete stored signal feature set that is a set of stored signal features of the stored signal, an incomplete stored signal feature set that is a set of stored signal features of the incomplete stored signal, and the stored additional information A stored latent variable extracting unit that extracts a stored latent variable that is a variable for describing a relationship between a signal and additional information from a stored additional information feature set that is a set of features, the complete stored signal feature set, A topic model, which is a model describing the relationship between a signal and additional information, is extracted from an incompletely stored signal feature set, the stored additional information feature set, and a stored latent variable set that is a set of the stored latent variables. A topic model learning unit; when an input signal is input, an input signal feature extraction unit that extracts an input signal feature that is a vector expressing characteristics of the input signal; and the input signal feature is given to the topic model, and a signal recognition unit for selecting the additional information related to the input signal from the accumulation additional information collection, the storage latent variable extraction unit, the said complete stored signal feature set incomplete A latent variable space generation unit that generates a latent variable space, which is a multidimensional vector space in which accumulated latent variables exist, from the accumulated signal feature set and the accumulated additional information feature set, the complete accumulated signal feature set, and the accumulated additional information A fully stored latent variable set extracting unit that extracts a fully stored latent variable set that is a stored latent variable set corresponding to the fully stored signal feature set using the feature set and the latent variable space; and the complete stored signal feature set And the incompletely stored signal feature set and the latent variable space are used to extract an incompletely stored latent variable set that is a stored latent variable set corresponding to the incompletely stored signal feature set, and the incompletely stored latent variable set the full accumulation latent variables semi-supervised signals, characterized in Rukoto a storage latent variable set extraction unit that sets and combined to form an overall accumulated latent variable set recognition search instrumentation and It is a position.

Another embodiment of the present invention is a semi-supervised signal recognition / search method used in a semi-supervised signal recognition / search apparatus constructed using a computer, wherein the accumulated signal feature extraction unit is information that describes a signal. A complete accumulation signal that is an element of a complete accumulation signal set that is a set of signals to which information is given in advance, and an incomplete accumulation signal that is an element of an incomplete accumulation signal set that is a set of signals to which no additional information is given. A step of extracting a stored signal feature that is a vector expressing the characteristics of each signal from each of them, and a stored additional information feature extraction unit is an element of a stored additional information set that is a set of additional information given to the complete stored signal A step of extracting a stored additional information feature that is a vector expressing the characteristics of the stored additional information from each of the stored additional information, and a stored latent variable extracting unit, A complete accumulated signal feature set that is a set of accumulated signal features of all accumulated signals, an incomplete accumulated signal feature set that is a set of accumulated signal features of the incomplete accumulated signal, and an accumulation addition that is a set of accumulated additional information features. Extracting an accumulated latent variable that is a variable for describing the relationship between a signal and additional information from the information feature set; and a topic model learning unit comprising the complete accumulated signal feature set and the incompletely accumulated signal feature. Extracting a topic model that is a model describing the relationship between a signal and additional information from the set, the stored additional information feature set, and a stored latent variable set that is a set of the stored latent variables; When the input additional information is input, the information feature extraction unit extracts an input additional information feature that is a vector expressing the characteristics of the input additional information, and a signal search unit , Given the input additional information wherein the topic models, the full accumulation signal set and viewed including the steps of selecting a signal related to the input additional information from the incomplete stored signal set, extracting the accumulated latent variables The latent variable space generation unit generates a latent variable space that is a multidimensional vector space in which accumulated latent variables exist from the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the accumulated additional information feature set. A generating step, and a fully stored latent variable set extraction unit uses the complete stored signal feature set, the stored additional information feature set, and the latent variable space to store a stored latent variable set corresponding to the fully stored signal feature set. Extracting a complete accumulated latent variable set, and the accumulated latent variable set extracting unit includes the complete accumulated signal feature set and the incompletely accumulated signal feature. Using the set and the latent variable space, an incompletely stored latent variable set that is a stored latent variable set corresponding to the incompletely stored signal feature set is extracted, and the incompletely stored latent variable set and the completely stored latent variable set are extracted. and forming the entire storage latent variable set to match the door is half teacher signal recognition search method characterized by including Mukoto.

Another embodiment of the present invention is a semi-supervised signal recognition / search method used in a semi-supervised signal recognition / search apparatus constructed using a computer, wherein the accumulated signal feature extraction unit is information that describes a signal. A complete accumulation signal that is an element of a complete accumulation signal set that is a set of signals to which information is given in advance, and an incomplete accumulation signal that is an element of an incomplete accumulation signal set that is a set of signals to which no additional information is given. A step of extracting a stored signal feature that is a vector expressing the characteristics of each signal from each of them, and a stored additional information feature extraction unit is an element of a stored additional information set that is a set of additional information given to the complete stored signal A step of extracting a stored additional information feature that is a vector expressing the characteristics of the stored additional information from each of the stored additional information, and a stored latent variable extracting unit, A complete accumulated signal feature set that is a set of accumulated signal features of all accumulated signals, an incomplete accumulated signal feature set that is a set of accumulated signal features of the incomplete accumulated signal, and an accumulation addition that is a set of accumulated additional information features. Extracting an accumulated latent variable that is a variable for describing the relationship between a signal and additional information from the information feature set; and a topic model learning unit comprising the complete accumulated signal feature set and the incompletely accumulated signal feature. Extracting a topic model that is a model describing a relationship between a signal and additional information from the set, the stored additional information feature set, and a stored latent variable set that is a set of the stored latent variables; and an input signal When the input signal is input to the feature extraction unit, the feature extraction unit extracts an input signal feature that is a vector representing the characteristics of the input signal; and the signal recognition unit includes the input signal. Given symptom into the topic models, look including the step of selecting the additional information related to the input signal from the storage additional information set, extracting the accumulated latent variables are latent variables space generation unit, the complete Generating a latent variable space that is a multidimensional vector space in which accumulated latent variables exist from the accumulated signal feature set, the incomplete accumulated signal feature set, and the accumulated additional information feature set; and a completely accumulated latent variable set extraction unit Extracting a complete stored latent variable set that is a stored latent variable set corresponding to the complete stored signal feature set using the complete stored signal feature set, the stored additional information feature set, and the latent variable space; The accumulated latent variable set extraction unit uses the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the latent variable space to generate an incomplete accumulated signal. Extracting an incompletely accumulated latent variable set that is an accumulated latent variable set corresponding to the feature set and combining the incompletely accumulated latent variable set and the complete accumulated latent variable set to form an entire accumulated latent variable set; which is a semi-supervised signal recognition search method characterized by including Mukoto.

One embodiment of the present invention is a complete stored signal set in which a stored signal feature extraction unit is a set of signals in which additional information that is information for explaining a signal is given in advance to a computer as a semi-teacher signal recognition and search device. Stored signal that is a vector that expresses the characteristics of each signal from the complete stored signal that is an element of the incomplete stored signal that is an element of the incompletely stored signal set that is a set of signals to which no additional information is given A feature extraction step, and a storage additional information feature extraction unit that calculates the characteristics of the storage additional information from each of the storage additional information that is an element of the storage additional information set that is a set of additional information added to the complete storage signal. A step of extracting a stored additional information feature which is a vector to be expressed; The relationship between the signal and the additional information is described from the incomplete stored signal feature set that is a set of stored signal features of the incomplete stored signal and the stored additional information feature set that is a set of the stored additional information features Extracting a stored latent variable that is a variable for the purpose, and a topic model learning unit, wherein the complete stored signal feature set, the incomplete stored signal feature set, the stored additional information feature set, and the stored latent variable A step of extracting a topic model, which is a model describing the relationship between a signal and additional information, from the accumulated latent variable set that is a set, and when the input additional information feature extraction unit receives the input additional information, Extracting an input additional information feature that is a vector expressing the characteristics of the input additional information, and a signal search unit gives the input additional information feature to the topic model, All the stored signal set, and to execute and selecting a signal related to the input additional information from the incomplete stored signal set, extracting the accumulated latent variables are latent variables space generation unit, the full accumulation signal Generating a latent variable space, which is a multidimensional vector space in which accumulated latent variables exist, from the feature set, the incompletely accumulated signal feature set, and the accumulated additional information feature set; and a complete accumulated latent variable set extraction unit, Using the complete stored signal feature set, the stored additional information feature set, and the latent variable space, extracting a fully stored latent variable set that is a stored latent variable set corresponding to the complete stored signal feature set; A latent variable set extraction unit uses the complete stored signal feature set, the incomplete stored signal feature set, and the latent variable space to generate an incomplete stored signal feature. Extracting an incompletely stored latent variable set that is a stored latent variable set corresponding to the set, and combining the incompletely stored latent variable set and the complete stored latent variable set to form an entire stored latent variable set is a program characterized Rukoto is.

  According to the present invention, the accumulated latent variable extraction unit describes the relationship between the signal and the additional information from both the signal associated with the additional information (language information) and the signal not associated with the additional information. By extracting the accumulated latent variable, the relationship between the signal and the additional information can be accurately learned even when only a small amount of the signal associated with the additional information can be used. That is, when learning the relationship between a signal and additional information, a signal that is not associated with additional information is used at the same time, which is higher than when only a small number of signals associated with additional information are used. The relationship between the signal and the additional information can be learned with accuracy. Thereby, it is possible to search and recognize signals with higher accuracy.

It is a block diagram which shows the function structure of the semi-teacher signal recognition search apparatus by 1st Embodiment. It is a block diagram which shows the function structure of the accumulation latent variable extraction part by 1st Embodiment. It is a block diagram which shows the function structure of the topic model learning part by 1st Embodiment. It is a flowchart which shows the procedure of the semi-teacher signal search process by 1st Embodiment. It is a flowchart which shows the procedure of the accumulation latent variable extraction process in the accumulation latent variable extraction part by 1st Embodiment. It is a flowchart which shows the procedure of the topic model learning process in the topic model learning part by 1st Embodiment. It is a block diagram which shows the function structure of the semi-teacher signal recognition search apparatus by 2nd Embodiment. It is a flowchart which shows the procedure of the semi-teacher signal recognition process by 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
First, the words used below are defined as follows.
The additional information is information (text label) describing the target signal, and generally refers to a signal of a different type from the target signal. The types of signals used here include, for example, still images, moving images (video), acoustic signals (speech, music, environmental sounds, etc.), text, biological signals (electroencephalograms, myoelectric signals, etc.), geographic information (GPS acquisition) Information, location name, etc.) and device information (camera model number, encoding codec, etc.). In the following, the expression “stored image” is used to explain the case where an image is used as a target signal and text is used as additional information. However, since it can be replaced with a signal as described above, It may be read as “accumulated signal”. A completely accumulated image (signal) is an image (signal) to which additional information is given in advance. An incompletely stored image (signal) is an image (signal) to which no additional information is given. Further, a completely accumulated image (signal) set G _c = {g ₁ , g ₂ ,..., G _N } is a set of N (N is an integer of 1 or more) sheets of completely accumulated images. The incompletely stored image (signal) set G _I = {g _{N + 1} , g _{N + 2} ,..., G _{N + Nx} } is a set of Nx (Nx is an integer of 1 or more) sheets of incompletely stored images. In many cases, N << Nx (N is sufficiently smaller than Nx). Also, full storage image set G _c and imperfect accumulated image set G _I and the combined accumulated image (signal) to the set G. The accumulated additional information set W1 = {w ₁ , w ₂ ,..., W _N } is a set of additional information given to the image (signal).

A. First Embodiment FIG. 1 is a block diagram showing a functional configuration of a semi-teacher signal recognition / retrieval apparatus 10 according to a first embodiment.
Semi-supervised signal recognition search apparatus 10, complete with stored images set G _c incomplete stored image set G _I and the storage additional information set W1 receives the input additional information, stored images an image (signal) associated with the input additional information It is an apparatus constructed from, for example, a computer that searches and outputs from the set G. The input additional information is additional information serving as a key for searching for an image (signal). The semi-teacher signal recognition / retrieval device 10 includes an accumulated signal feature extracting unit 1, an accumulated additional information feature extracting unit 2, an accumulated latent variable extracting unit 3, a topic model learning unit 4, an input additional information feature extracting unit 5, And a signal search unit 6.

Stored signal feature extraction unit 1, a complete and a stored image set G _c and imperfect accumulated image set G _I as an input, a full storage image set G _c elements in a complete stored image and an incomplete storage image set G _I elements From each of the incompletely stored images, stored image (signal) features, which are vectors representing the characteristics of each image, are extracted. The accumulated signal feature extraction unit 1 then performs a completely accumulated image (signal) feature set that is a set of completely accumulated image (signal) features and an incompletely accumulated image (signal) feature that is a set of incompletely accumulated image (signal) features. The set is output to the accumulated latent variable extraction unit 3. The completely stored image feature is a stored image feature of the completely stored image, and the incompletely stored image feature is a stored image feature of the incompletely stored image.

Specifically, if the accumulated image is a still image, the accumulated signal feature extraction unit 1 first performs SURF (Speed-up robust features; for example, H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” CVIU, vol. 110, no. 3, pp. 346-359, 2008.) is used to detect image feature points. Subsequently, the accumulated signal feature extraction unit 1 quantizes the local feature amount calculated for the feature point, and expresses each quantized feature amount as a histogram. As a result, the accumulated signal feature extraction unit 1 performs the complete accumulated image feature set X _c = {x ₁ , x ₂ ,..., X _N } and the incomplete accumulated image feature set X _I = {x _{N + 1} , x _{N + 2} ,. xN _{+ Nx} } is obtained. Elements x _n (n = 1, 2, 3,..., Nx) of the image feature set are histograms.

  If the stored image is a moving image, a histogram generated from a time-series signal as in the “time-series active search (TAS) method (Japanese Patent No. 30653314)” may be used as a feature amount. Further, when an acoustic signal is used instead of the stored image, MFCC (Mel-frequency cepstral coefficients), chroma feature amount, or the like can be used. Further, in the case of geographic information, information such as longitude, latitude, and time acquired by GPS can be used as a feature amount as it is. In the case of text, it will be described in the explanation of the accumulated load information feature extraction unit 2 described later.

  In the present embodiment, the accumulated signal feature extraction unit 1 obtains the accumulated image feature from the quantized histogram of the local feature amount, but instead, for example, a color histogram, digital of each small region in the image Low frequency component of cosine transform, low frequency component or high frequency component of Haar wavelet, high-order local autocorrelation feature (for example, N. Otsu and T. Kurita “A new scheme for practical flexible and intelligent vision systems,” Proc. IAPR Workshop on Computer Vision, pp.431-435, 1988) can also be used. In addition, the above-described SURF is used for feature point extraction and local feature amount calculation. Instead, for example, SIFT (for example, D. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal other methods such as of Computer Vision, Vol. 60, No. 2, pp. 91-110, 2004) can also be used.

  The accumulated additional information feature extraction unit 2 receives the accumulated additional information set W1 as input, and stores accumulated additional information features that are vectors representing the characteristics of the accumulated additional information from each of the accumulated additional information that is an element of the accumulated additional information set W1. The stored additional information feature set Y, which is a set of stored additional information features, is output to the stored latent variable extraction unit 3. The method of extracting the accumulated additional information feature set Y is not particularly limited, but in the present embodiment, the following three examples will be described.

[First embodiment]
The accumulated additional information feature is a binary vector that represents the presence or absence of a word (language label) included in the text label. That is, the accumulated additional information feature is a vector having the same number of dimensions as the total number of words to be considered, and each dimension of the vector corresponds to a word. For convenience, the word is expressed using the corresponding dimension index in the vector. The accumulated additional information feature extraction unit 2 sets the i-th dimension of the accumulated additional information feature to 1 if the word i is included in the accumulated additional information, and 0 otherwise.

[Second Embodiment]
The word co-occurrence vector Y = {y ₁ , y ₂ ,..., Y _N } is used as the accumulated additional information feature. Storing the additional information feature extraction unit 2, the d-th element y _n of word cooccurrence vector y _n, the _d, and d-th number of words included in the text labels w _n.

[Third embodiment]
Use LSA (Lentent Semantic Analysis). Considering a matrix W in which the vector derived by the method of the first or second embodiment is a column vector, the accumulated additional information feature extraction unit 2 performs singular value decomposition on this matrix W, so that W = UΣV ^T ( Obtain orthonormal matrices U and V and diagonal matrix Σ satisfying V ^T (transpose matrix of V). The singular values of W are arranged in the diagonal component of Σ. At this time, it is assumed that the U and V columns are appropriately rearranged so that the value of Σ is in descending order from the upper left (in descending order). Next, the accumulated additional information feature extraction unit 2 uses a submatrix ΣDy of Σ upper left Dy × _Dy and a submatrix U _Dy extracted up to the left Dy column of U for a predetermined _Dy , storing the additional information, wherein _{y n (n = 1,2, ···} , n) and obtained by _{^{y n = U Dy T Σ Dy}} w n.

The accumulated latent variable extraction unit 3 receives the complete accumulated image feature set _Xc , the incompletely accumulated image feature set _XI, and the accumulated additional information feature set Y as inputs, and from these feature sets, an image (signal) and an additional An accumulated latent variable set Z = {z ₁ , z ₂ ,..., Z _{Nx + N} }, which is a set of variables for describing information relationships, is extracted, and this accumulated latent variable set is extracted to the topic model learning unit 4. Output. Details of the processing in the accumulated latent variable extraction unit 3 will be described later.

Topic model learning unit 4, a full storage image feature set X _C, and an incomplete storage image feature set X _I, and the accumulated additional information feature set Y, and inputs the accumulation latent variable set Z, from these sets, images A topic model, which is a model that describes the relationship between and the additional information, is learned (extracted), and the topic model is output. Details of the processing in the topic model learning unit 4 will be described later.

  When the input additional information is input, the input additional information feature extraction unit 5 extracts an input additional information feature that is a vector representing the characteristics of the input additional information, and outputs the extracted input additional information feature to the signal search unit 6. To do. The method for extracting the input additional information feature is the same as the method for extracting the stored additional information feature in the stored additional information feature extraction unit 2.

  The signal search unit 6 receives the accumulated image set G, the topic model, and the input additional information feature as input, and gives the input additional information feature to the topic model, whereby an input addition that is an image related to the input additional information from the accumulated image set G is obtained. An information related image is selected, and an input additional information related image set which is a set of the input additional information related images is output. Details of the processing in the signal search unit 6 will be described later.

Next, details of the processing in the accumulated latent variable extraction unit 3 will be described. FIG. 2 is a block diagram showing a functional configuration of the accumulated latent variable extraction unit 3 according to the present embodiment.
The accumulated latent variable extraction unit 3 includes a function f _x : R ^Dx → R ^Dz (a function that generates a set of Dz-dimensional real vectors from a set of Dx-dimensional real vectors), and a function f _xy : R ^Dx × R ^Dy → R ^Dz A set (f _x , f _xy ) of (a function that generates a set of Dz-dimensional real vectors from a set of Dx-dimensional real vectors and a set of Dy-dimensional real vectors) is obtained from (X, Y), and these functions Is used to generate Z from (X, Y). Here, X is the accumulation image feature set that matches the full storage image feature set X _c and incomplete accumulated image feature set X _I. Dx is the dimension of the stored image feature, Dy is the dimension of the stored additional information feature, and Dz is the dimension of the stored latent variable. In the present embodiment, the accumulated latent variable set is extracted using a method obtained by improving canonical correlation analysis, which is a kind of multivariate analysis. The accumulated latent variable extracting unit 3 includes a latent variable space generating unit 31, a complete accumulated latent variable set extracting unit 32, and an accumulated latent variable set extracting unit 33.

The latent variable space generation unit 31 is a latent variable that is a multidimensional vector space in which accumulated latent variables exist from the completely accumulated image feature set _Xc , the incompletely accumulated image feature set _XI, and the accumulated additional information feature set Y. Create a space. The generation method of the latent variable space is not particularly limited, but in the present embodiment, the latent variable space generation unit 31 is derived by solving the generalized eigenvalue problem given by the following equation (1). The generalized eigenvalue problem is a problem for obtaining an eigenvalue λ and an eigenvector a that satisfy the equation (1).

  Here, a is a (Dx + Dy) -dimensional vector. However, B in Formula (1) is represented by the following Formula (2).

  Moreover, C in Formula (1) is represented by following Formula (3).

Where ^T means transpose (vector or matrix). I _d represents a d × d unit matrix. In addition, the character which attached | subjected the hat (^) to the character upper part in a type | formula shows and describes a hat (^) before a character in a sentence. B and each element hat constituting the _{^{C (^) S (C)}} xx, hat _{^{(^) S (C) yy}} , hat _{^{(^) S (C) xy}} , hat _{^{(^) S (C) yx}} , hat ( ^) S _xx and hat (^) S _yy are expressed by the following equations (4) to (9).

  Here, β is a real number given in advance. For the solution of the generalized eigenvalue problem, as a reference, Hardon, Szedmak and Shawe-Taylor “Canonical correlation analysis: An overview with applications to learning methods,” Neural Computation, Vol.16, Issue 12, pp.2639- 2664, December 2004.

  Solving the generalized eigenvalue problem shown in Equation (1) to obtain a set of a predetermined number of eigenvalues and eigenvectors, or a set of a minimum number of eigenvalues and eigenvectors whose sum of eigenvalues exceeds a predetermined threshold. Alternatively, the latent variable space can be determined by obtaining the number of eigenvalues and eigenvectors determined by the method using the Bartlett test. Regarding the Bartlett test, there is M.S. Bartlett “THE STATISTICAL SIGNIFICANCE OF CANONICAL CORRELATIONS,” Biometrika, Vol. 32 pp. 29-37, 1941.

However, Dz is the number of extracted eigenvalues and eigenvectors and satisfies Dz ≦ min (Dx, Dy). Each eigenvector a _d (d = 1, 2,..., Dz) can be decomposed into a leading Dx-dimensional vector a _{x, d} and a subsequent Dy-dimensional vector a _{y, d} . Thereby, the latent variable space generation unit 31 outputs a matrix (A _x , A _y ) having eigenvectors obtained by decomposing the latent variable space as columns. Here, A _x = {a _{x, 1} , a _{x, 2} , a _{x, 3} ,..., A _{x, Dx} }, and A _y = {a _{y, 1} , a _{y, 2} , a _{y, 3} ,..., A _{y, Dy} }.

The complete storage latent variable set extraction unit 32 uses the complete storage image feature set _Xc , the storage additional information feature set Y, and the latent variable space A to store a storage latent variable set corresponding to the complete storage image feature set _{Xc. Is} extracted as a fully accumulated latent variable set Zc. Specifically, first, the complete accumulation latent variable set extraction unit 32 determines f _xy by the following equation (10). In addition, a character with “˜” in the upper part of the character in the formula is indicated by “˜” before the character in the sentence.

Here, Λ is a diagonal matrix having the d-th eigenvalue λ _d (d = 1, 2,..., Dz) as the d-th diagonal component in the solution of Equation (1). M _x and M _y are Dz-dimensional square matrices that satisfy M _x M _y ^T = Λ and whose spectral norm is smaller than 1. ˜Λ is expressed by the following equation (11).

Then, the complete accumulation latent variable set extraction unit 32 calculates each element z _n (n = 1, 2,... N) of the complete accumulation latent variable set Z _c from the following equation (12), whereby the complete accumulation latent variable is calculated. A set _Zc is extracted.

Accumulated latent variable set extraction unit 33, a full storage image feature set X _c, and an incomplete storage image feature set X _I, by using the latent variable space A, the storage potential corresponding to the incompletely stored image feature set X _I extracting the incomplete storage latent variable set Z _I is a variable set. Then, the accumulated latent variable set extraction unit 33 combines the complete accumulated latent variable set Z _c and the incomplete accumulated latent variable set Z _I to form the entire accumulated latent variable set Z. Specifically, first, accumulated latent variable set extraction unit 33 defines the _{f x} by the following equation (13).

Then, the accumulated latent variable set extraction unit 33 calculates each element z _n (n = N + 1, N + 2,... N + Nx) of the incomplete accumulated latent variable set Z _I from the following equation (14), thereby obtaining the incomplete accumulated latent potential. to extract a set of variables _{Z I.}

The accumulated latent variable set extraction unit 33, a storage latent variable set Z together with full accumulation latent variable set Z _c and incomplete accumulated latent variable set Z _I, and outputs the accumulated latent variable set Z.

Next, details of processing in the topic model learning unit 4 will be described. FIG. 3 is a block diagram showing a functional configuration of the topic model learning unit 4 according to the present embodiment.
The topic model learning unit 4 includes a latent variable model learning unit 41, a signal / latent variable relationship model learning unit 42, an initial additional information / latent variable relationship model learning unit 43, an additional information / latent variable relationship model updating unit 44, The additional information / latent variable relationship model correcting unit 45 and the convergence determining unit 46 are included.

  The latent variable model learning unit 41 receives the accumulated latent variable set Z, learns (extracts) a latent variable model that is a model describing the stochastic structure of the accumulated latent variable itself, and outputs the latent variable model. The learning method of the latent variable model is not particularly limited, but in the present embodiment, the following three examples will be described. In either case, the occurrence probability p (z) of the accumulated latent variable z is adopted as the latent variable model.

[First embodiment]
The latent variable model learning unit 41 calculates a latent variable model p (z) by multipoint approximation of the following equation (15) using the accumulated latent variable set Z. Here, Kz (z) is a probability density function with z as a parameter. For example, Dirac delta function or normal distribution can be used as Kz. That is, the latent variable model learning unit 41 extracts a latent variable model p (z) by superimposing probability density functions using the accumulated latent variable z as a parameter.

[Second Embodiment]
The latent variable model learning unit 41 considers the probabilistic structure of the canonical correlation analysis and calculates the latent variable model p (z) by the following equation (16). However, the function N is a normal distribution in which the first argument is an average and the second argument is a covariance matrix.

[Third embodiment]
The latent variable model learning unit 41 calculates a latent variable model p (z) by a mixed normal distribution represented by the following equation (17).

However, the number of Mz is mixed distribution, omega _m is the weight of the m-th distribution ^{_{(Σ m = 1 Mz ω m}} = 1), ¯z m average of the m-th distribution, sigma _zm is the m-th distribution Represents the variance. If Mz is determined in advance, ω _m , ￣z _m , and Σ _zm can be obtained by the EM algorithm.

  The signal / latent variable relationship model learning unit 42 receives a stored image feature set X and a stored latent variable set Z, and uses these sets as a signal that describes a relationship between the stored image and the stored latent variable.・ Learning (extracting) the latent variable relationship model and outputting the signal / latent variable relationship model. The learning method of the signal / latent variable relationship model is not particularly limited, but in the present embodiment, the following two examples will be described. In either case, the conditional occurrence probability p (x | z) of the stored image feature x when the stored latent variable z is given is adopted as the signal / latent variable relationship model.

[First embodiment]
The signal / latent variable relationship model learning unit 42 calculates a signal / latent variable relationship model p (x | z _n ) for each element z _n of the accumulated latent variable set by the following equation (18). That is, the signal / latent variable relationship model learning unit 42 extracts a signal / latent variable relationship model individually for each accumulated latent variable z _n and based on a probability density function using the accumulated image feature x as a parameter.

  Here, ~ z is an accumulated latent variable obtained by converting the accumulated image feature x by the accumulated latent variable extracting unit 3, and Kx (z) is a probability density function having z as a parameter. As Kx, for example, Dirac delta function or normal distribution can be used.

[Second Embodiment]
The signal / latent variable relationship model learning unit 42 calculates the signal / latent variable relationship model p (x | z) by the following equation (19) in consideration of the probabilistic structure of the canonical correlation analysis.

  The initial additional information / latent variable relationship model learning unit 43 is a model that uses the stored additional information feature set Y and the stored latent variable set Z as inputs and describes the relationship between the additional information and the stored latent variables using these sets. A certain initial additional information / latent variable relationship model is learned (extracted), and this initial additional information / latent variable relationship model is output. The learning method of the initial additional information / latent variable relationship model is not particularly limited, but in the present embodiment, the following three embodiments will be described. In either case, the conditional occurrence probability p (y | z) of the additional information feature y when the accumulated latent variable z is given is adopted as the initial additional information / latent variable relationship model.

[First embodiment]
A method similar to that of the signal / latent variable relationship model learning unit 42 can be used.

[Second Embodiment]
The initial additional information / latent variable relationship model learning unit 43 calculates the initial additional information / latent variable relationship model p (y | z _n ) from the following equations (20) to (23) for each element z _n of the accumulated latent variable set. calculate. That is, the initial additional information / latent variable relationship model learning unit 43 extracts an additional information / latent variable relationship model individually for each accumulated latent variable z _n and based on a probability density function using additional information as a parameter. .

Here, mu is a constant satisfying 0 ≦ μ ≦ 1, y n , d is the d component of the accumulated additional information feature y _n. Ky (y) is a probability density function with y as a parameter. For example, a Kronecker delta function or a multinomial distribution can be used. That is, the above relational expression first assumes that each text label occurs independently (formula (20)), and the occurrence probability of each text label is expressed as an empirical distribution (formula ( 21) Ky (equivalent to y _d −y _{n, d} )) and the empirical distribution of text labels in all samples (corresponding to M _d / M in equation (21)) are generated by mixing at a mixing ratio μ. Means that.

[Third embodiment]
The initial additional information / latent variable relationship model learning unit 43 calculates the initial additional information / latent variable relationship model p (y | z) by the following equation (24) in consideration of the probabilistic structure of the canonical correlation analysis.

  The additional information / latent variable relationship model updating unit 44 includes an initial additional information / latent variable relationship model (or additional information / latent variable relationship model corrected by the additional information / latent variable relationship model correcting unit 45) and a stored additional information feature. From the set Y and the stored latent variable set Z, the (initial) additional information / latent variable relation model is updated in consideration of the distribution of the stored latent variables corresponding to the incompletely stored image feature set, and new additional information / latency Output as a variable relation model.

  Hereinafter, in order to simplify the notation, a matrix notation represented by the following equation (25) is introduced.

P _{m, n} is an (m, n) component of the matrix P and is expressed by the following equation (26).

F _{n, d} is an (n, d) component of the matrix F and is expressed by the following equation (27).

Here, the matrix F is the additional information / latent variable relationship model. Further, the initial additional information / latent variable relationship model obtained by the initial additional information / latent variable relationship model learning unit 43 is F ^(0), and the additional information / latent variable relationship model updated for the kth time is F ^(k) . . The additional information / latent variable relationship model updating unit 44 updates the additional information / latent variable relationship model by the following equation (28), and outputs F ^(k) as a new additional information / latent variable relationship model.

  When there are a large number of stored images, there is a problem that the matrix P becomes too large. In order to avoid this, the following method can be taken.

  First, among the elements of the matrix P calculated by the equation (26), the element having a small value is forcibly rewritten to 0. As a criterion for selecting an element to be rewritten, for example, a method of selecting a predetermined number of elements for each row of the matrix P or selecting an element that falls below a predetermined threshold can be considered. In many embodiments, each element of the matrix P is determined by the distance between two vectors in the latent variable space, so in this case, determining which elements to leave unmodified is in the latent variable space. Equivalent to performing a neighborhood search or range search at.

  Next, a new matrix P hat that retains only the remaining elements without being replaced with 0 is considered, and this is used instead of the matrix P to calculate equation (28).

  The original matrix P requires an area about the square of the number of stored images to hold it, while the new matrix P hat reduces the area required to hold it to the same order as the number of stored images. be able to.

  The additional information / latent variable relationship model correcting unit 45 stores the additional information / latent variable relationship model updated by the additional information / latent variable relationship model updating unit 44 and the stored additional information feature set Y. The additional information / latent variable relationship model is corrected so as to hold the feature information to a certain degree (predetermined value), and is output as a new additional information / latent variable relationship model. The additional information / latent variable relationship model F can be described by the following equation (29).

Here, F ^(k) _L is the additional information / latent variable relationship model of the N × Dy matrix corresponding to the completely accumulated image, and F ^(k) _U is the Nx × Dy matrix of the incompletely accumulated image. Represents additional information / latent variable relationship model. The additional information / latent variable relationship model correcting unit 45 corrects the additional information / latent variable relationship model for the completely stored image in the additional information / latent variable relationship model F by the following equation (30).

Here, T is a diagonal matrix with the (n, n) component being t _n (0 <t _n ≦ 1, n = 1, 2,..., N), It is a regularization parameter for deciding whether to correct the additional information / latent variable relationship model with emphasis on the degree. The value of the matrix T can be freely determined in advance.

The convergence determination unit 46 includes the additional information / latent variable relationship model corrected by the additional information / latent variable relationship model update unit 44 and the additional information / latent variable relationship model correction unit 45, and the additional information / latent variable relationship before correction. The model (the output immediately before the additional information / latent variable relationship model correction unit 45) is compared. If the convergence determination unit 46 determines that the additional information / latent variable relationship model has converged, the convergence determination unit 46 outputs the additional information / latent variable relationship model and ends the process, while determining that the additional information / latent variable relationship model has not converged. For example, the processing shifts to the additional information / latent variable relationship model update unit 44, and the updating / correction processing of the additional information / latent variable relationship model is repeated again. At this time, the convergence determination unit 46 calculates a matrix norm of the corrected additional information / latent variable relationship model F ^{(k + 1)} and the corrected additional information / latent variable relationship model F ^(k−1) , and this matrix norm If the difference is within a predetermined threshold value, it is determined that it has converged, and if it is greater than the threshold value, it is determined that it has not converged.

  The topic model learning unit 4 integrates the latent variable model, the signal / latent variable relationship model, and the additional information / latent variable relationship model into a topic model, and outputs the topic model.

Next, details of processing in the signal search unit 6 will be described. The signal search unit 6 gives an input additional information feature to the topic model to extract an image related to the input additional information, and selects an image having high compatibility from the stored image set G as an input additional information related image. The selection method of the input additional information related image is not particularly limited, but in the present embodiment, the following method will be described. First, the signal search unit 6 sets the posterior probability of the stored image feature x when the input additional information feature y _g is given by the following equation (31).

  Then, the signal search unit 6 calculates this posterior probability for each element of the accumulated image feature set X, selects an accumulated image corresponding to an image feature that exceeds a predetermined threshold, and inputs and adds these image sets. Let it be an information image set. Alternatively, the signal search unit 6 may form a set of input additional information images by selecting a predetermined number of stored images corresponding to image features having a large posterior probability.

When the topic model has been generated for each element z _n of the accumulated latent variable set Z in the topic model learning unit 4, the posterior probability of the above equation (31) can be rewritten by the following equation (32). it can.

Next, with reference to FIGS. 4 to 6, the semi-teacher signal search process in the semi-teacher signal recognition / search apparatus 1 according to the present embodiment will be described. FIG. 4 is a flowchart showing the procedure of the semi-teacher signal search process.
First, in step S1, the stored signal feature extraction unit 1 extracts the complete stored image set and the image feature of each image from each is an element of the incomplete storage image set, full storage image feature set X _C and incomplete storage An image feature set _XI is generated. Next, in step S2, the accumulated additional information feature extraction unit 2 extracts accumulated additional information features from each accumulated additional information that is an element of the accumulated additional information set, and generates an accumulated additional information feature set Y. Next, in step S3, the accumulation latent variable extractor 3, and a full storage image feature set X _C and incomplete accumulated image feature set X _I and accumulating additional information feature set Y, extracts the accumulated latent variable set Z. Subsequently, in step S4, the topic model learning unit 4, full accumulation and image feature set X _C, and an incomplete storage image feature set X _I, the accumulated additional information feature set Y, based on the accumulated latent variable set Z Then, a topic model which is an image / additional information relation model is extracted. Next, in step S5, the input additional information feature extraction unit 5 extracts the input additional information feature from the input additional information. Finally, in step S6, the signal search unit 6 selects the input additional information related image set from the accumulated image set by giving the input additional information feature to the topic model.

FIG. 5 is a flowchart showing a procedure of accumulated latent variable extraction processing in the accumulated latent variable extracting unit 3 according to the present embodiment.
First, in step S31, the latent variable space generation unit 31, and a full storage image feature set X _C and incomplete accumulated image feature set X _I and accumulating additional information feature set Y, to generate a latent variable space w. Next, in step S32, fully accumulated latent variable set extraction unit 32, full storage image feature set using the X _C and accumulating additional information feature set Y and latent variable space w, a complete accumulation latent variable set Z _C extraction To do. Finally, in step S33, accumulated latent variable set extraction unit 33, by using the latent variable space w completely stored image feature set X _C and incomplete accumulated image feature set X _I, incomplete storage latent variable set Z _I Are extracted to form the entire accumulated latent variable set Z.

FIG. 6 is a flowchart showing the procedure of the topic model learning process in the topic model learning unit 4 according to the present embodiment.
First, in step S41, the latent variable model learning unit 41 learns a latent variable model p (z) from the accumulated latent variable set Z. Next, in step S42, the signal / latent variable relationship model learning unit 42 learns the signal / latent variable relationship model p (x | z) from the accumulated image feature set X and the accumulated latent variable set Z. Next, in step S43, the initial additional information / latent variable relationship model learning unit 43 learns the initial additional information / latent variable relationship model p (y | z) from the accumulated additional information feature set Y and the accumulated latent variable set Z. To do.

  Next, in step S 44, the additional information / latent variable relationship model updating unit 44 adds the initial additional information / latent variable relationship model p (y | z) or the additional information / latent variable relationship model correction unit 45 corrected. Update information / latent variable relationship model. Next, in step S45, the additional information / latent variable relationship model correcting unit 45 corrects the additional information / latent variable relationship model. In step S46, the convergence determination unit 46 compares the corrected additional information / latent variable relationship model with the uncorrected additional information / latent variable relationship model to determine whether or not the convergence has occurred. If it is determined that it has converged, the convergence determination unit 46 outputs the corrected additional information / latent variable relationship model and ends the process. On the other hand, when it determines with not having converged, it transfers a process to step S44.

Thus, according to the present embodiment, the accumulated latent variable extraction unit 3 describes the relationship between the image and the additional information from both the image associated with the additional information and the image not associated with the additional information. By extracting the storage latent variables for this purpose, even when only a small amount of the image associated with the additional information can be used, the relationship between the image and the additional information can be learned with high accuracy. That is, when learning the relationship between an image and additional information, an image that is not associated with additional information is used at the same time, which is higher than when only a small number of images associated with additional information are used. The relationship between the image and the additional information can be learned with accuracy. Thereby, an image related to the input additional information can be searched with higher accuracy.
Further, the additional information / latent variable relationship model update unit 44 and the additional information / latent variable relationship model correction unit 45 update and correct the additional information / latent variable relationship model until the additional information / latent variable relationship model converges, thereby providing more accurate additional information.・ Latent variable relationship model can be output.

B. Second Embodiment Next, a second embodiment of the present invention will be described.
FIG. 7 is a block diagram showing a functional configuration of the semi-teacher signal recognition / retrieval apparatus 20 according to the second embodiment.
Semi-supervised signal recognition retrieval device 20 is completely stored image set G _c and incomplete stored image set G _I and the storage additional information set W1 and the input image (signal) as input, storage additional information the additional information related to the input image It is a device constructed from, for example, a computer that retrieves and outputs from a set. The semi-teacher signal recognition / retrieval device 20 includes an accumulated signal feature extracting unit 11, an accumulated additional information feature extracting unit 12, an accumulated latent variable extracting unit 13, a topic model learning unit 14, an input signal feature extracting unit 17, a signal And a recognition unit 18. Since the processes of the accumulated signal feature extracting unit 11, the accumulated additional information feature extracting unit 12, the accumulated latent variable extracting unit 13, and the topic model learning unit 14 are the same as those in the first embodiment, the description thereof is omitted.

  When an input image is input, the input signal feature extraction unit 17 extracts an input image (signal) feature that is a vector representing the characteristics of the input image, and outputs the input image feature. The input image feature extraction method is the same as the stored image feature extraction method in the stored signal feature extraction unit 11.

The signal recognizing unit 18 receives the input image feature, the topic model, and the accumulated additional information set as input, and gives the input image feature to the topic model, whereby the input image that is additional information given to the image having high similarity to the input image. (Signal) related additional information is extracted, and this input image related additional information is output. The method for selecting the input image related additional information is not particularly limited, but in the present embodiment, the following method will be described. First, the signal recognition unit 18 sets the posterior probability of the additional information feature y when the input image feature _xg is given by the following equation (33).

  Then, the signal recognizing unit 18 calculates this posterior probability for each element of the accumulated additional information feature set Y. Next, the signal recognizing unit 18 calculates an additional information feature y hat that maximizes the posterior probability by the following equation (34).

  Note that the additional information feature y hat with the maximum posterior probability shown above generally does not become a binary vector. When the additional information can be expressed as a vector expressing the presence or absence of a word included in the text label, the signal recognition unit 18 has a predetermined threshold value among the elements of the additional information feature y hat with the maximum posterior probability. Elements beyond the limit are selected, and language labels corresponding to the respective elements are collected and used as input image related additional information. Alternatively, the signal recognition unit 18 may select a predetermined number of elements having a large value.

When the topic model is generated for each element z _n of the accumulated latent variable set Z in the topic model learning unit 4, the posterior probability of the above equation (33) can be rewritten by the following equation (35). it can.

Next, the semi-teacher signal recognition process in the semi-teacher signal recognition / retrieval apparatus 20 according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing the procedure of the semi-teacher signal recognition process.
First, in step S101, the stored signal feature extraction unit 11 extracts the full storage image sets and the image feature of each image from each is an element of the incomplete storage image set, full storage image feature set X _C and incomplete storage An image feature set _XI is generated. Next, in step S102, the accumulated additional information feature extraction unit 12 extracts accumulated additional information features from each accumulated additional information that is an element of the accumulated additional information set, and generates an accumulated additional information feature set Y. Next, in step S103, the accumulation latent variable extraction unit 13, and a full storage image feature set X _C and incomplete accumulated image feature set X _I and accumulating additional information feature set Y, extracts the accumulated latent variable set Z. Then, based on in step S104, the topic model learning unit 14, and a fully stored images feature set X _C, and an incomplete storage image feature set X _I, the accumulated additional information feature set Y, and a storage latent variable set Z Thus, a topic model that is an image / additional information relation model is extracted. Next, in step S105, the input signal feature extraction unit 17 extracts an input image feature from the input image. Finally, in step S106, the signal recognizing unit 18 selects input image related additional information from the accumulated additional information set by giving an input image feature to the topic model.

  Thus, according to the present embodiment, the accumulated latent variable extraction unit 13 describes the relationship between the image and the additional information from both the image associated with the additional information and the image not associated with the additional information. By extracting the storage latent variables for this purpose, even when only a small amount of the image associated with the additional information can be used, the relationship between the image and the additional information can be learned with high accuracy. That is, when learning the relationship between an image and additional information, an image that is not associated with additional information is used at the same time, which is higher than when only a small number of images associated with additional information are used. The relationship between the image and the additional information can be learned with accuracy. Thereby, more appropriate additional information can be given to the input image.

Also, a program for realizing each step shown in FIG. 4 to FIG. 6 or FIG. 8 is recorded on a computer-readable recording medium, and the program recorded on this recording medium is read into a computer system and executed. Thus, the semi-teacher signal search process or the semi-teacher signal recognition process may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to
For example, in this embodiment, a stored image set and a stored additional information set are input, but the semi-teacher signal recognition / retrieval devices 10 and 20 include a storage unit, and the stored image set and the stored additional information set are stored in this storage unit. It may be stored in advance.

  DESCRIPTION OF SYMBOLS 1,11 ... Accumulated signal feature extraction part 2,12 ... Storage additional information feature extraction part 3,13 ... Storage latent variable extraction part 4,14 ... Topic model learning part 5 ... Input additional information feature extraction part 6 ... Signal search part 17 ... Input signal feature extraction unit 18 ... Signal recognition unit 10,20 ... Semi-teacher signal recognition search device 31 ... Latent variable space generation unit 32 ... Complete accumulation latent variable set extraction unit 33 ... Storage latent variable set extraction unit 41 ... Latent variable model Learning unit 42: Signal / latent variable relationship model learning unit 43 ... Initial additional information / latent variable relationship model learning unit 44 ... Additional information / latent variable relationship model update unit 45 ... Additional information / latent variable model correction unit 46 ... Convergence determining unit

Claims (5)

A complete accumulation signal that is an element of a complete accumulation signal set that is a set of signals to which additional information, which is information describing the signal, is given in advance, and an incomplete accumulation signal set that is a set of signals to which no additional information is given An accumulated signal feature extraction unit that extracts an accumulated signal feature that is a vector expressing the characteristics of each signal from each of the incomplete accumulated signals that are elements;
Accumulated additional information that extracts a stored additional information feature that is a vector expressing the characteristics of the accumulated additional information from each of the accumulated additional information that is an element of the accumulated additional information set that is a set of additional information assigned to the complete accumulated signal A feature extraction unit;
A complete accumulated signal feature set that is a set of accumulated signal features of the complete accumulated signal, an incomplete accumulated signal feature set that is a set of accumulated signal features of the incomplete accumulated signal, and an accumulation that is a set of the accumulated additional information features. An accumulated latent variable extracting unit that extracts an accumulated latent variable that is a variable for describing the relationship between the signal and the additional information from the additional information feature set;
Describe the relationship between the signal and additional information from the complete stored signal feature set, the incomplete stored signal feature set, the stored additional information feature set, and the stored latent variable set that is a set of the stored latent variables. A topic model learning unit that extracts a topic model that is a model to be
When input additional information is input, an input additional information feature extraction unit that extracts an input additional information feature that is a vector expressing the characteristics of the input additional information;
A signal search unit that gives the input additional information feature to the topic model, and selects a signal related to the input additional information from the complete accumulation signal set and the incomplete accumulation signal set , and
The accumulated latent variable extraction unit includes:
A latent variable space generation unit that generates a latent variable space that is a multidimensional vector space in which a stored latent variable exists, from the complete stored signal feature set, the incomplete stored signal feature set, and the stored additional information feature set;
A fully stored latent variable that extracts a fully stored latent variable set that is a stored latent variable set corresponding to the complete stored signal feature set using the complete stored signal feature set, the stored additional information feature set, and the latent variable space. A set extraction unit;
Using the complete stored signal feature set, the incomplete stored signal feature set, and the latent variable space, an incomplete stored latent variable set that is a stored latent variable set corresponding to the incomplete stored signal feature set is extracted, An accumulated latent variable set extraction unit that forms the entire accumulated latent variable set by combining the incompletely accumulated latent variable set and the complete accumulated latent variable set;
Semi-supervised signal recognition retrieval apparatus according to claim Rukoto equipped with. A complete accumulation signal that is an element of a complete accumulation signal set that is a set of signals to which additional information, which is information describing the signal, is given in advance, and an incomplete accumulation signal set that is a set of signals to which no additional information is given An accumulated signal feature extraction unit that extracts an accumulated signal feature that is a vector expressing the characteristics of each signal from each of the incomplete accumulated signals that are elements;
Accumulated additional information that extracts a stored additional information feature that is a vector expressing the characteristics of the accumulated additional information from each of the accumulated additional information that is an element of the accumulated additional information set that is a set of additional information assigned to the complete accumulated signal A feature extraction unit;
A complete accumulated signal feature set that is a set of accumulated signal features of the complete accumulated signal, an incomplete accumulated signal feature set that is a set of accumulated signal features of the incomplete accumulated signal, and an accumulation that is a set of the accumulated additional information features. An accumulated latent variable extracting unit that extracts an accumulated latent variable that is a variable for describing the relationship between the signal and the additional information from the additional information feature set;
Describe the relationship between the signal and additional information from the complete stored signal feature set, the incomplete stored signal feature set, the stored additional information feature set, and the stored latent variable set that is a set of the stored latent variables. A topic model learning unit that extracts a topic model that is a model to be
When an input signal is input, an input signal feature extraction unit that extracts an input signal feature that is a vector expressing the characteristics of the input signal;
A signal recognition unit that gives the input signal characteristics to the topic model and selects additional information related to the input signal from the accumulated additional information set ; and
The accumulated latent variable extraction unit includes:
A latent variable space generation unit that generates a latent variable space that is a multidimensional vector space in which a stored latent variable exists, from the complete stored signal feature set, the incomplete stored signal feature set, and the stored additional information feature set;
A fully stored latent variable that extracts a fully stored latent variable set that is a stored latent variable set corresponding to the complete stored signal feature set using the complete stored signal feature set, the stored additional information feature set, and the latent variable space. A set extraction unit;
Using the complete stored signal feature set, the incomplete stored signal feature set, and the latent variable space, an incomplete stored latent variable set that is a stored latent variable set corresponding to the incomplete stored signal feature set is extracted, An accumulated latent variable set extraction unit that forms the entire accumulated latent variable set by combining the incompletely accumulated latent variable set and the complete accumulated latent variable set;
Semi-supervised signal recognition retrieval apparatus according to claim Rukoto equipped with. A semi-teacher signal recognition / search method used in a semi-teacher signal recognition / search apparatus constructed using a computer,
The accumulated signal feature extraction unit is a complete accumulated signal that is an element of a completely accumulated signal set that is a set of signals preliminarily given additional information that is information describing the signal, and a set of signals that are not given additional information. Extracting a stored signal feature, which is a vector representing the characteristics of each signal, from each of the incomplete stored signals that are elements of a certain incomplete stored signal set;
The accumulated additional information feature extraction unit is a vector that expresses the characteristics of accumulated additional information from each of the accumulated additional information that is an element of the accumulated additional information set that is a set of additional information added to the complete accumulated signal. Extracting information features;
An accumulated latent variable extraction unit includes a complete accumulated signal feature set that is a set of accumulated signal features of the completely accumulated signal, an incomplete accumulated signal feature set that is a set of accumulated signal features of the incomplete accumulated signal, and the accumulation addition. Extracting a stored latent variable that is a variable for describing a relationship between a signal and additional information from a stored additional information feature set that is a set of information features;
A topic model learning unit includes a signal and additional information from the complete stored signal feature set, the incomplete stored signal feature set, the stored additional information feature set, and a stored latent variable set that is a set of the stored latent variables. Extracting a topic model, which is a model that describes the relationship between and
When the input additional information feature extraction unit receives the input additional information, the step of extracting the input additional information feature that is a vector expressing the characteristics of the input additional information;
Signal search unit provides the input additional information wherein the topic models, look including the step of selecting a signal related to the input additional information from the full accumulation signal set and the incomplete stored signal set,
Extracting the accumulated latent variable comprises:
A latent variable space generation unit generates a latent variable space, which is a multidimensional vector space in which accumulated latent variables exist, from the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the accumulated additional information feature set. When,
A complete storage latent variable set extraction unit uses the complete storage signal feature set, the stored additional information feature set, and the latent variable space to store a complete storage latency that is a storage latent variable set corresponding to the complete storage signal feature set. Extracting a set of variables;
The accumulated latent variable set extraction unit uses the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the latent variable space, and is an incomplete accumulation that is an accumulated latent variable set corresponding to the incomplete accumulated signal feature set. Extracting a latent variable set and combining the incompletely stored latent variable set and the complete stored latent variable set to form an entire stored latent variable set;
Semi-supervised signal recognition search method comprising including Mukoto a. A semi-teacher signal recognition / search method used in a semi-teacher signal recognition / search apparatus constructed using a computer,
The accumulated signal feature extraction unit is a complete accumulated signal that is an element of a completely accumulated signal set that is a set of signals preliminarily given additional information that is information describing the signal, and a set of signals that are not given additional information. Extracting a stored signal feature, which is a vector representing the characteristics of each signal, from each of the incomplete stored signals that are elements of a certain incomplete stored signal set;
The accumulated additional information feature extraction unit is a vector that expresses the characteristics of accumulated additional information from each of the accumulated additional information that is an element of the accumulated additional information set that is a set of additional information added to the complete accumulated signal. Extracting information features;
An accumulated latent variable extraction unit includes a complete accumulated signal feature set that is a set of accumulated signal features of the completely accumulated signal, an incomplete accumulated signal feature set that is a set of accumulated signal features of the incomplete accumulated signal, and the accumulation addition. Extracting a stored latent variable that is a variable for describing a relationship between a signal and additional information from a stored additional information feature set that is a set of information features;
A topic model learning unit includes a signal and additional information from the complete stored signal feature set, the incomplete stored signal feature set, the stored additional information feature set, and a stored latent variable set that is a set of the stored latent variables. Extracting a topic model, which is a model that describes the relationship between and
An input signal feature extraction unit that, when an input signal is input, extracts an input signal feature that is a vector representing the characteristics of the input signal;
Signal recognition unit provides the input signal, wherein the topic models, look including the step of selecting the additional information related to the input signal from the storage additional information set,
Extracting the accumulated latent variable comprises:
A latent variable space generation unit generates a latent variable space, which is a multidimensional vector space in which accumulated latent variables exist, from the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the accumulated additional information feature set. When,
A complete storage latent variable set extraction unit uses the complete storage signal feature set, the stored additional information feature set, and the latent variable space to store a complete storage latency that is a storage latent variable set corresponding to the complete storage signal feature set. Extracting a set of variables;
The accumulated latent variable set extraction unit uses the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the latent variable space, and is an incomplete accumulation that is an accumulated latent variable set corresponding to the incomplete accumulated signal feature set. Extracting a latent variable set and combining the incompletely stored latent variable set and the complete stored latent variable set to form an entire stored latent variable set;
Semi-supervised signal recognition search method comprising including Mukoto a. In a computer as a semi-teacher signal recognition search device,
The accumulated signal feature extraction unit is a complete accumulated signal that is an element of a completely accumulated signal set that is a set of signals preliminarily given additional information that is information describing the signal, and a set of signals that are not given additional information. Extracting a stored signal feature, which is a vector representing the characteristics of each signal, from each of the incomplete stored signals that are elements of a certain incomplete stored signal set;
The accumulated additional information feature extraction unit is a vector that expresses the characteristics of accumulated additional information from each of the accumulated additional information that is an element of the accumulated additional information set that is a set of additional information added to the complete accumulated signal. Extracting information features;
An accumulated latent variable extraction unit includes a complete accumulated signal feature set that is a set of accumulated signal features of the completely accumulated signal, an incomplete accumulated signal feature set that is a set of accumulated signal features of the incomplete accumulated signal, and the accumulation addition. Extracting a stored latent variable that is a variable for describing a relationship between a signal and additional information from a stored additional information feature set that is a set of information features;
A topic model learning unit includes a signal and additional information from the complete stored signal feature set, the incomplete stored signal feature set, the stored additional information feature set, and a stored latent variable set that is a set of the stored latent variables. Extracting a topic model, which is a model that describes the relationship between and
When the input additional information feature extraction unit receives the input additional information, the step of extracting the input additional information feature that is a vector expressing the characteristics of the input additional information;
A signal search unit providing the input additional information feature to the topic model, and selecting a signal related to the input additional information from the complete accumulation signal set and the incomplete accumulation signal set , and
Extracting the accumulated latent variable comprises:
A latent variable space generation unit generates a latent variable space, which is a multidimensional vector space in which accumulated latent variables exist, from the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the accumulated additional information feature set. When,
A complete storage latent variable set extraction unit uses the complete storage signal feature set, the stored additional information feature set, and the latent variable space to store a complete storage latency that is a storage latent variable set corresponding to the complete storage signal feature set. Extracting a set of variables;
The accumulated latent variable set extraction unit uses the complete accumulated signal feature set, the incomplete accumulated signal feature set, and the latent variable space, and is an incomplete accumulation that is an accumulated latent variable set corresponding to the incomplete accumulated signal feature set. Extracting a latent variable set and combining the incompletely stored latent variable set and the complete stored latent variable set to form an entire stored latent variable set;
Program characterized Rukoto allowed to run.

Images