JP7583337B2 - Learning device, autoencoding device, learning method, autoencoding method, and program - Google Patents

Description

The present invention relates to a learning device, a self-encoding device, a learning method, a self-encoding method and a program.

In data compression using deep learning, there are two types of quantization processing: scalar quantization and vector quantization, with vector quantization providing higher compression performance. Attempts at vector quantization include those described in the following non-patent literature 1 and non-patent literature 2. The technology described in non-patent literature 1 is a technology called VQVAE. In the technology described in non-patent literature 1, end-to-end learning is not performed, and the distribution of feature vectors is assumed to be uniform, and optimization of encoding and decoding and optimization of representative vectors are performed separately. The technology in non-patent literature 2 is a technology called Soft to Hard. In the technology in non-patent literature 2, the quantization processing is approximated using a softmax function, and end-to-end learning is performed by approximating the occurrence probability of representative vectors using a histogram.

Ali Razavi, et.al.,"Generating Diverse High-Fidelity Images with VQ-VAE-2,",2019. Eirikur Agustsson, et.al.,"Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations,",2017.

However, conventional techniques such as the technique described in Non-Patent Document 1 and the technique described in Non-Patent Document 2 have problems with obtaining self-encoding processing using vector quantization. The problems include, for example, stability problems, processing performance problems, and problems such as an explosive increase in the amount of calculations or memory usage when the dimension of the vector is large or when encoding at a high rate. Thus, conventional techniques may require a large burden to obtain self-encoding processing using vector quantization. As a result, there are cases where the burden required for self-encoding using vector quantization is large, such as when self-encoding using vector quantization cannot be realized due to a lack of computer resources.

In view of the above circumstances, the present invention aims to provide a technology that reduces the burden required for self-encoding using vector quantization.

One aspect of the present invention is a learning device that includes a learning unit that updates the encoding and decoding processes in a self-encoding process using vector quantization, the self-encoding process using main data features, which are features to be self-encoded, and auxiliary features, which are features of the main data features, by learning, and that performs entropy coding on the result of vector quantizing the main data features and entropy coding on the result of scalar quantizing the auxiliary features. In the learning process, the learning unit executes a main data side probability estimation process that estimates the occurrence probability of each element of a tensor that indicates the main data features, and the main data side probability estimation process estimates the occurrence probability using the result of integrating a probability density function that is parametrized using an integral region that divides a vector space in which representative vectors are arranged in a lattice shape, a region that includes one lattice point of the vector space, and has a shape of a hypercuboid.

One aspect of the present invention includes an auto-encoding target acquisition unit that acquires an object of auto-encoding, and a learning unit that updates, by learning, the encoding and decoding processes in the auto-encoding process using vector quantization, the auto-encoding process using main data features that are features of the object of auto-encoding and auxiliary features that are features of the main data features, and entropy coding the result of vector quantizing the main data features and entropy coding the result of scalar quantizing the auxiliary features. In the learning, the learning unit updates each element of a tensor that indicates the main data features. and a self-encoding execution unit that performs a main data side probability estimation process to estimate the occurrence probability of an element, the main data side probability estimation process estimating the occurrence probability using the result of integrating a probability density function that is parameterized over a region that divides a vector space in which representative vectors are arranged in a lattice pattern, the region including one lattice point of the vector space, and the region having a shape of a hypercuboid as an integration region, and that performs self-encoding by vector quantization of the target acquired by the self-encoding target acquisition unit using a learned encoding process and a learned decoding process obtained using a learning device.

One aspect of the present invention is a self-encoding process using vector quantization, which uses main data features, which are features to be self-encoded, and auxiliary features, which are features of the main data features, and which performs entropy coding on the result of vector quantizing the main data features and entropy coding on the result of scalar quantizing the auxiliary features. The self-encoding process includes a learning step for updating the encoding and decoding processes by learning, and the learning step executes a main data side probability estimation process for estimating the occurrence probability of each element of a tensor indicating the main data features in the learning. The main data side probability estimation process is a learning method that estimates the occurrence probability using the result of integrating a probability density function that is parametrized using an integral region that divides a vector space in which representative vectors are arranged in a lattice shape, includes one lattice point of the vector space, and has a shape of a hypercuboid.

One aspect of the present invention includes an auto-encoding target acquisition step for acquiring an object for auto-encoding, and a learning unit for updating, by learning, the encoding and decoding processes in the auto-encoding process using vector quantization, which uses main data features that are features of the object for auto-encoding and auxiliary features that are features of the main data features, and which performs entropy coding on the result of vector quantizing the main data features and entropy coding on the result of scalar quantizing the auxiliary features, and the learning unit updates, by learning, the encoding and decoding processes in the auto-encoding process using main data features that are features of the object for auto-encoding and auxiliary features. a main data side probability estimation process for estimating an occurrence probability, the main data side probability estimation process estimating the occurrence probability using a result of integrating a probability density function that is parameterized over a region that divides a vector space in which representative vectors are arranged in a lattice pattern, the region including one lattice point of the vector space, and the region having a shape of a hypercuboid, as an integration region; and a self-encoding execution step for performing self-encoding by vector quantization of the target acquired in the self-encoding target acquisition step using a learned encoding process and a learned decoding process obtained using a learning device.

One aspect of the present invention is a program for causing a computer to function as the above-mentioned learning device.

One aspect of the present invention is a program for causing a computer to function as the above-mentioned self-encoding device.

The present invention makes it possible to reduce the burden required for self-encoding using vector quantization.

FIG. 1 is an explanatory diagram illustrating an overview of a learning device according to an embodiment. FIG. 2 is an explanatory diagram for explaining LatticeVQ in the embodiment. FIG. 4 is a first explanatory diagram illustrating an example of adding noise according to the embodiment. FIG. 11 is a second explanatory diagram illustrating an example of adding noise in the embodiment. 5A and 5B are explanatory diagrams illustrating the relationship between a cumulative distribution function and occurrence probability in the embodiment. FIG. 4 is an explanatory diagram for explaining hyper-rectangular parallelepiped division in the embodiment. FIG. 4 is a diagram showing an example of a processing flow executed by a learning unit in the embodiment. FIG. 1 is a first explanatory diagram illustrating an overview of a self-encoding device according to an embodiment. FIG. 2 is a second explanatory diagram illustrating an overview of the self-encoding device according to the embodiment. 10 is a flowchart showing an example of a flow of a process executed by an encoder in the embodiment. 10 is a flowchart showing an example of a flow of a process executed by a decoder in the embodiment. FIG. 2 is a diagram showing an example of a hardware configuration of a learning device according to an embodiment. FIG. 2 is a diagram showing an example of the configuration of a control unit included in the learning device according to the embodiment. FIG. 2 is a diagram showing an example of a hardware configuration of a self-encoding device according to an embodiment. FIG. 2 is a diagram showing an example of the configuration of a control unit included in the self-encoding device according to the embodiment.

(Embodiment)
FIG. 1 is an explanatory diagram for explaining an overview of a learning device 1 according to an embodiment. The learning device 1 includes a learning unit 10. The learning unit 10 performs learning so as to improve the performance of self-encoding using vector quantization of data expressed by tensors. The performance of self-encoding is evaluated by the smallness of the RD cost (D+λR), which is the sum of the error D between the original data and the restored data and the code amount R of the data weighted by the Lagrangian constant λ. The smaller the RD cost, the better the RD performance. The learning means machine learning. The learning is, for example, deep learning. As is well known, the self-encoding of data means the compression of data.

The learning unit 10 includes a learning network 100 and an optimization unit 113. The learning network 100 is a neural network. The learning network 100 includes a main data acquisition unit 101, a main data side encoding unit 102, an auxiliary data side encoding unit 103, a main data side noise adding unit 104, an auxiliary data side noise adding unit 105, an auxiliary data side probability estimating unit 106, an auxiliary data side decoding unit 107, an auxiliary entropy acquisition unit 108, a main data side probability estimating unit 109, a main entropy acquisition unit 110, a main data side decoding unit 111, and a reconstruction error calculation unit 112. The optimization unit 113 updates the learning network 100 based on the output of the learning network 100, as will be described in detail later.

The main data acquisition unit 101 acquires data expressed in tensors as main data. Data expressed in tensors is, for example, image data. Data expressed in tensors may also be, for example, time series data of audio of one or more channels. Hereinafter, the data acquired by the main data acquisition unit 101 is referred to as main data.

The main data side encoding unit 102 executes a main data feature acquisition process. The main data feature acquisition process is a process of encoding the main data. Since encoding is a process of acquiring information indicating the features of the object of encoding, encoding is a process of acquiring information indicating the features.

Therefore, the encoding of main data by the main data side encoding unit 102 is a process of acquiring the features of the main data. Therefore, the main data feature acquisition process is a process of acquiring the main data features. The main data features are encoded main data. The contents of the main data feature acquisition process are updated through learning. In other words, the contents of the process performed by the main data side encoding unit 102 are updated through learning.

The auxiliary data side encoding unit 103 further encodes the main data features. As described above, encoding is a process of obtaining information indicating features, so encoding of the main data features by the auxiliary data side encoding unit 103 is a process of obtaining the features of the main data features. Therefore, hereinafter, information obtained by further encoding the encoded main data is referred to as auxiliary features. In other words, auxiliary features are information indicating the features of the main data features. Since auxiliary features are information obtained by encoding main data features, the entropy of auxiliary features is information with smaller entropy than main data features.

Hereinafter, the process of encoding the main data features is referred to as the auxiliary feature acquisition process. The content of the auxiliary feature acquisition process is updated through learning. In other words, the content of the process performed by the auxiliary data side encoding unit 103 is updated through learning.

The contents of the auxiliary feature acquisition process are updated through learning so that the auxiliary features contain more information in the main data statistics information. The main data statistics information is information indicating the statistics of each probability distribution to which the values of each element of the tensor representing the main data follow. The statistics of the probability distribution are, for example, the scatter. The statistics of the probability distribution may be not only the scatter but also a pair of the scatter and a representative value.

As explained in the fields of information processing and information theory, data generally appears according to a probability distribution, such as the probability distribution of the occurrence of each Roman character in an English sentence. The values of each element of a tensor representing the main data also follow a probability distribution.

The main data side noise addition unit 104 executes a noise-added main data feature acquisition process. The noise-added main data feature acquisition process is a process that executes a vector noise addition process on the main data feature. The vector noise addition process is a process that processes a K-dimensional vector (hereinafter referred to as a "noise-added vector") having a predetermined number K (K is an integer equal to or greater than 2) of elements as a processing object, and adds noise to the processing object. Therefore, the noise-added main data feature acquisition process is a process that acquires information in which noise has been added to the main data feature (hereinafter referred to as a "noise-added main data feature").

Specifically, the vector to which noise is added is a K-dimensional vector contained in a tensor that represents the main data features. A K-dimensional vector contained in a tensor means a vector whose k-th element (k is an integer between 1 and K) is the k-th element of K consecutive elements of all elements of the tensor.

The number of elements K of a vector to which noise is added is the number of elements that are mapped to one code by vector quantization by a device that performs vector quantization using the results of learning by the learning unit 10. Hereinafter, the results of learning by the learning unit 10 are referred to as network learning results. Therefore, when K elements are collectively mapped to one code by vector quantization using the results of network learning, the number of elements of a vector to which noise is added is K. Note that when K elements are collectively mapped to one code by quantization, the resulting code is an index indicating a K-dimensional vector.

Therefore, the number of elements K of the vector to which noise is added is a predetermined number. Note that vector quantization of data is encoding of data, so vector quantization using the network learning results means encoding of data using the network learning results.

Here we will explain how to add noise, but before explaining how to add noise we will explain LatticeVQ.

<LatticeVQ>
Vector quantization requires a representative vector. A lattice is a set of lattice points in a vector space. LatticeVQ is vector quantization in which representative vectors are arranged in a lattice pattern in a vector space. That is, LatticeVQ is This is vector quantization that satisfies the condition that representative vectors are arranged in a lattice pattern in a vector space. LatticeVQ is known to have better RD performance than scalar quantization, except under certain conditions.

Figure 2 is an explanatory diagram explaining LatticeVQ in an embodiment. More specifically, Figure 2 is an explanatory diagram explaining LatticeVQ when the number of elements of the noise-added vector is 2.

FIG. 2 is an example of the _A2 lattice. In the case of two dimensions, one type of lattice is the _A2 lattice. In the case of eight dimensions, one type of lattice is the _E8 lattice. In the case of 24 dimensions, one type of lattice is the Reach lattice. These are LatticeVQ lattices that maximize RD performance for a uniform distribution.

Hereinafter, a space in which the representative vectors are arranged in a lattice pattern will be referred to as a lattice space. In other words, the representative vectors are located at each lattice point of the lattice space. Figure 2 is also a diagram showing an example of a lattice space. In the example of Figure 2, the lattice space is two-dimensional, but if the representative vectors are K-dimensional, the lattice space is also K-dimensional.

<Adding noise>
An example of adding noise will now be described with reference to Fig. 3 and Fig. 4, taking the case where the number of elements is two as an example. Fig. 3 is a first explanatory diagram illustrating an example of adding noise in the embodiment. Fig. 4 is a second explanatory diagram illustrating an example of adding noise in the embodiment.

When adding noise, first generate multiple random K-dimensional vectors that are uniformly distributed within a (K-1)-dimensional sphere circumscribing the lattice unit area of the origin lattice (hereinafter referred to as the "target lattice unit area"), and place them in the lattice space. For example, in the example of Figure 4, when adding noise, first generate until the number of internal points in Figure 4 is at least 1. The lattice unit areas are each area that results from dividing the lattice space into multiple areas so that the division conditions are satisfied.

The division conditions are that the size and shape of each region are the same, and that each region contains one lattice point of the lattice space. Therefore, a lattice unit region is a region that divides a vector space in which representative vectors are arranged in a lattice pattern, and contains one lattice point of the vector space.

Note that a (K-1)-dimensional sphere is a circle if K = 2, a sphere if K = 3, and a hypersphere if K is 4 or more. Region B1 in Figure 3 is an example of a target lattice unit region.

In the examples of Figures 3 and 4, the lattice unit region is a Voronoi region. A Voronoi region is each of the regions obtained by dividing a metric space such as a lattice space into multiple regions by Voronoi division.

The noise points in Figure 3 are an example of samples placed within the circumscribing circle of the target lattice unit area. Specifically, the process of placing samples within lattice space is the process of obtaining coordinates within the lattice space. Since a lattice unit area in lattice space is an area within lattice space, the process of placing samples within the circumscribing circle of any one lattice unit area is the process of obtaining coordinates within that circumscribing circle.

Next, of the placed noise points, noise points located within the target lattice unit area (hereinafter referred to as "internal points") and noise points located outside the target lattice unit area (hereinafter referred to as "exclusion points") are distinguished as shown in the example of Figure 4. Specifically, the process of distinguishing between internal points and exclusion points is a process of determining, for each noise point, whether the noise point has coordinates within the target lattice unit area or coordinates outside the target lattice unit area, based on its coordinates. Specifically, the determination process is a process in which the noise points are vector quantized using equation (11), and points that match the origin lattice are determined to have coordinates within the area.

Next, one of the multiple noise points located within the target lattice unit area is randomly selected. The selected noise point is then added to one noise-added target vector. Specifically, the addition process refers to a process in which the noise-added target vector and a position vector indicating the selected noise point are added.

In this way, the position vector indicating the selected noise point is a quantity determined by a random number, and is therefore a type of noise. And since the position vector indicating the selected noise point is expressed as a vector, the position vector indicating the selected noise point is noise expressed as a vector. Therefore, hereafter, the position vector indicating the selected noise point will be referred to as the noise vector. In this way, noise is added to the noise-added vector.

Returning to the explanation of FIG. 1, the auxiliary data-side noise addition unit 105 executes a noise-added auxiliary feature acquisition process. The noise-added auxiliary feature acquisition process is a process of adding noise to the auxiliary feature. In other words, the noise-added auxiliary feature acquisition process is a process of acquiring information in which scalar noise has been added to the auxiliary feature (hereinafter referred to as "noisy auxiliary feature"). More specifically, the auxiliary data-side noise addition unit 105 adds scalar noise to each element of the tensor representing the auxiliary feature. The scalar noise is, for example, uniform noise between -1/2 and 1/2.

The auxiliary data side probability estimation unit 106 executes auxiliary data side probability estimation processing. The auxiliary data side probability estimation processing is processing for estimating auxiliary data side probability based on noise-added auxiliary features. The auxiliary data side probability is information indicating the occurrence probability of each element of a tensor representing an auxiliary feature. Information on the probability distribution of each element of a tensor representing an auxiliary feature is used to estimate the occurrence probability of each element of a tensor representing an auxiliary feature.

When the probability distribution of each element of the tensor indicating the auxiliary feature is a predetermined probability distribution, the auxiliary data side probability estimation unit 106 acquires the predetermined probability distribution by reading it from a predetermined storage device such as the storage unit 14 described below. Then, the auxiliary data side probability estimation unit 106 estimates the occurrence probability of each element of the tensor indicating the auxiliary feature based on the acquired probability distribution. Note that the predetermined probability distribution is a probability distribution expressed using, for example, a cumulative distribution function.

When the probability distribution of each element of the tensor representing the auxiliary feature is expressed by a parameterized auxiliary feature cumulative distribution function, the auxiliary data side probability estimation unit 106 estimates the occurrence probability of each element of the tensor representing the auxiliary feature based on the parameterized auxiliary feature cumulative distribution function. The parameterized auxiliary feature cumulative distribution function is a parameterized function that indicates the probability distribution of each element of the tensor representing the auxiliary feature. Specifically, the parameters of the parameterized auxiliary feature cumulative distribution function are parameters that change according to statistics that represent the probability distribution of each element of the tensor representing the auxiliary feature.

The parameter values of the parameterized auxiliary feature cumulative distribution function are updated by learning. In other words, the content of the processing performed by the auxiliary data side probability estimation unit 106 is updated by learning.

The parameterized cumulative distribution function is, for example, a parameterized sigmoid function or a softplus function. As described above, the parameter values of the parameterized auxiliary feature cumulative distribution function are updated by learning, so the parameter values of the parameterized cumulative distribution function are updated by learning.

Here, the relationship between the cumulative distribution function and occurrence probability will be explained using an example where the random variable is one-dimensional. FIG. 5 is an explanatory diagram explaining the relationship between the cumulative distribution function and occurrence probability in an embodiment. Image G1 is an image showing an example of a probability density function showing the probability density of random variable q. Δ indicates the quantization step size. Image G1 shows that the value obtained by integrating the probability density function over a range Δ in the domain centered on the value q is the occurrence probability p(q) of the value q. In the one-dimensional case as in FIG. 5, the step size Δ is, for example, the size of the closed interval [-1/2, 1/2] (i.e., 1).

Image G2 is a cumulative distribution function cdf(q) obtained as a result of integrating the probability density function of image G1. In this way, the result of integrating the probability density function is expressed by a monotonically increasing function such as a sigmoid function. Note that the result of integrating the probability density function is expressed by a monotonically increasing function such as a sigmoid function regardless of the shape of the probability density function. More specifically, a cumulative distribution function is a function that satisfies the conditions cdf(-∞) = 0, cdf(∞) = 1, and the differential value of cdf(q) with respect to q is 0 or greater.

Image G2 shows that the probability of occurrence p(q) is equal to cdf(q+Δ/2)-cdf(q-Δ/2). Returning to the explanation of Figure 1.

The auxiliary data side decoding unit 107 executes an auxiliary feature decoding process. The auxiliary feature decoding process is a process in which information obtained based on an auxiliary feature is processed, and the process decodes the processed object. The processing object of the auxiliary feature decoding process executed by the auxiliary data side decoding unit 107 is the noise-added auxiliary feature. Hereinafter, information obtained by decode the noise-added auxiliary feature is referred to as auxiliary data. Therefore, the auxiliary data side decoding unit 107 obtains the auxiliary data by decoding the noise-added auxiliary feature.

As described above, the more the content of the auxiliary feature acquisition process is updated through learning, the smaller the RD cost becomes. Therefore, the more the content of the auxiliary feature acquisition process is updated, the greater the maximum amount of information of the main data statistical information that the auxiliary data can contain.

The content of the auxiliary feature decoding process is updated by learning so that the RD cost is reduced. In other words, the content of the process performed by the auxiliary data side decoding unit 107 is updated by learning.

The auxiliary entropy acquisition unit 108 acquires auxiliary entropy based on the auxiliary data side probability, which is the result of estimation by the auxiliary data side probability estimation unit 106. The auxiliary entropy is the entropy of the auxiliary feature.

The main data side probability estimation unit 109 executes a main data side probability estimation process. The main data side probability estimation process is a process for estimating the main data side probability based on the noise-added main data features and auxiliary data. The main data side probability is information indicating the occurrence probability of each element of a tensor indicating the main data features. Information on the probability distribution of each element of a tensor indicating the main data features is used to estimate the occurrence probability of each element of a tensor indicating the main data features.

When the probability distribution of each element of a tensor representing the main data features is expressed using a parametrized main data feature cumulative distribution function, the main data side probability estimation unit 109 estimates the occurrence probability of each element of the tensor representing the main data features based on the parametrized main data feature cumulative distribution function.

The parametrized main data feature cumulative distribution function is a parametrized cumulative distribution function that indicates the probability distribution of each element of a tensor that indicates the main data feature. The probability distribution is, for example, a Gaussian distribution.

The parameters of the parametrized main data feature cumulative distribution function are, specifically, statistics that represent the probability distribution of each element of the tensor that represents the main data feature. Therefore, in learning, the values indicated by the auxiliary data are used as the parameter values of the parametrized auxiliary feature cumulative distribution function.

For example, if the probability distribution is a Gaussian distribution, the auxiliary data obtained in the trained learning network 100 indicates the representative value and the degree of dispersion of the Gaussian distribution.

<Estimation of occurrence probability>
An example of the main data side probability estimation process will be described. More specifically, an example of the main data side probability estimation process when the above-mentioned LatticeVQ is used will be described. When vector quantization is performed, each vector of the main data is represented by one of the representative vectors. Therefore, the description of the main data side probability estimation process when LatticeVQ is used is, more specifically, a description of a process for estimating the occurrence probability of the representative vector when LatticeVQ is used. Hereinafter, the process for estimating the occurrence probability of the representative vector will be referred to as the representative vector occurrence probability estimation process. The representative vector occurrence probability estimation process is an example of the main data side probability estimation process.

<Representative Vector Occurrence Probability Estimation Process>
As described above, the main data side probability estimation process uses a parameterized main data feature cumulative distribution function. As described above, the parameterized main data feature cumulative distribution function is specifically a parameterized cumulative distribution function. As described above, a cumulative distribution function is a result of integrating a probability density function. Therefore, the representative vector occurrence probability estimation process is a process that uses a cumulative distribution function. And, the use of a cumulative distribution function in the representative vector occurrence probability estimation process means that the result of integrating a parametrized probability density function in a lattice unit domain is used.

As mentioned above, a Voronoi region is one of the lattice unit regions. Since the Voronoi tessellation is a well-known method of domain division, it is often used to obtain lattice unit regions. However, when the lattice space is two-dimensional, the shape of the Voronoi region is a hexagon, as mentioned above.

However, when integrating a function on a manifold, if the shape of the region indicating the integration domain is hexagonal, it is not necessarily easy to perform the integration. Note that "easy to perform the integration" means that the amount of calculation required to obtain the integration result with a precision equal to or higher than a predetermined precision is small.

Furthermore, when a manifold is higher than two-dimensional, integrating a function in the region obtained as a result of the Voronoi tessellation is even less easy than in the two-dimensional case. For example, in the four-dimensional case, it is often difficult to draw the shape of the Voronoi region in two-dimensional space, and it is not easy to integrate a function on such a manifold. Thus, it is not necessarily easy to integrate a function using the region obtained as a result of the Voronoi tessellation as the integration region.

Therefore, it may not be easy to obtain the parameterized cumulative distribution function used in the representative vector occurrence probability estimation process.

Therefore, in the representative vector occurrence probability estimation process, a parametrized cumulative distribution function obtained using the results of hypercube division is used instead of Voronoi division. Hypercube division is a process that divides lattice space into lattice unit regions whose shape is a hypercube. Note that a two-dimensional hypercube is a rectangle, and a three-dimensional hypercube is a rectangular prism.

Figure 6 is an explanatory diagram explaining hyper-rectangle division in an embodiment. More specifically, Figure 6 shows an example of the result of hyper-rectangle division for a two-dimensional lattice space, together with an example of the result of Voronoi division. The "true region" and "approximate region" in Figure 6 are both examples of lattice unit regions.

The "true region" is the region resulting from the Voronoi tessellation. That is, the "true region" is the Voronoi region. The "true region" is a hexagonal lattice unit region. The "approximate region" is the lattice unit region resulting from the hypercuboid tessellation.

The shape of the "approximation region" is a rectangular parallelepiped. Of the two types of division results shown in Fig. 6, an example of the result of the hyper rectangular parallelepiped division executed by the main data side probability estimation unit 109 is the result of division by the "approximation region". _S1 in Fig. 6 is the length of a side of the rectangular parallelepiped which is the shape of the "approximation region", and indicates the length of the side of the first dimension in the two-dimensional lattice space, and _S2 indicates the length of the side of the second dimension.

The length of the first dimension side means the length of the rectangle in one of the oblique spaces when a rectangle in a lattice space is projected onto two orthogonal oblique spaces. The length of the second dimension side means the length of the rectangle in the other oblique space. In the following explanation, the length of the nth dimension side of a hyper-rectangle in an N-dimensional lattice space means the length of the hyper-rectangle in the nth oblique space when the hyper-rectangle is projected onto N one-dimensional oblique spaces that are orthogonal. A hyper-rectangle in an oblique space is a straight line.

Incidentally, integrals with an integration domain that is two or more dimensions can be obtained by iterative integration. When integrating a function with an N-dimensional hypercuboid as its integration domain, iterative integration is performed by integrating the function N times. In the case of a hypercuboid, it is possible to perform iterative integration with the integration domain of each of the N integrals being the edges of the hypercuboid.

When performing an iterative integral in which the integration domain of each of the N integrals is the edge of a hypercube, each integral is not affected by the results of the other integrals. On the other hand, in the case of a hyperpolyhedron that is not a hypercube, each integral of the iterative integral is affected by the results of the other integrals. Therefore, when the shape of the lattice unit domain is a hypercube, it is easier to perform the integral than when the shape of the lattice unit domain is a hyperpolyhedron that is not a hypercube.

In the example of Figure 6, the "true domain" is hexagonal in shape and the "approximate domain" is rectangular in shape, so performing integration using the "approximate domain" as the integration domain is easier than integration using the "true domain" as the integration domain.

When obtaining a parametrized cumulative distribution function using the hyperrectangle obtained as a result of the hyperrectangle division as the integration domain, in each integral of the iterative integral, an integration of a one-dimensional probability density function that is not affected by other dimensions is performed.

In the learning network 100, the probability density function that is the subject of the iterative integration is a function that indicates a predetermined type of distribution, and is a parametrized function. The predetermined type of distribution is a Gaussian distribution. The parameter values of the probability density function are values that correspond to the auxiliary data. The parametrized cumulative distribution function is obtained by performing iterative integration on the probability density function, with the hyperrectangle obtained by the hyperrectangle division being the integration domain. In the learning network 100, the parameter values of the probability density function are values of the auxiliary data, so the parametrized cumulative distribution function obtained in this manner is a function that corresponds to the auxiliary data.

In the representative vector occurrence probability estimation process, the values of the auxiliary data are substituted into the parameters of the cumulative distribution function thus obtained in advance. In the representative vector occurrence probability estimation process, the cumulative distribution function thus obtained is used to estimate the occurrence probability of the representative vector.

In the representative vector occurrence probability estimation process, the occurrence probability of the representative vector is estimated by executing the processes shown in the following formulas (1) and (2). In formula (2), the cumulative distribution function cdf represents each cumulative distribution function obtained by each integral of the repeated integral. The result of integrating an m-dimensional (m is a natural number) cumulative distribution function for one dimension is an (m-1)-dimensional cumulative distribution function.

The right-hand side of equation (1) is an example of an occurrence probability obtained using a parametrized main data feature cumulative distribution function. Equation (1) is a function parametrized via the cumulative distribution function cfd of equation (2). Equation (2) is a cumulative distribution function obtained as a result of integrating the probability density function in the one-dimensional direction of the lattice unit area. Equation (1) is the product of the occurrence probabilities of each dimension obtained by equation (2). Therefore, equation (1) is an occurrence probability expressed using the result of integrating the probability density function over the entire lattice unit area.

i is an identifier for identifying each lattice point. j means each dimension of the lattice space. The left side of formula (1) indicates the occurrence probability of the representative vector. Since each lattice point indicates a representative vector, the occurrence probability p _i at each lattice point means the occurrence probability p _i of each representative vector. Δ _j means the length of the dimension represented by the identifier j of the lattice unit area. Since the shape and size of the lattice unit area are predetermined, Δ _j is a predetermined length.

Note that the y with a hat symbol in the following formula (3) means the noise-added main data feature. Hereinafter, the hat symbol A will be written as A^. Therefore, for example, the hat symbol y in the following formula (3) is y^.

Equation (1) indicates that the occurrence probability p _i is expressed as the product of the probabilities p _i [j] obtained for each dimension. If the shape of the lattice unit area is not a hypercuboid, the occurrence probability p _i is not expressed as a simple product of the probabilities p _i [j] obtained for each dimension. Therefore, since the shape of the lattice unit area is a hypercuboid, it is easy for the main data side probability estimation unit 109 to obtain the occurrence probability p _i as shown in equations (1) and (2).

Note that the above formulas (1) and (2) are obtained under the assumption that the covariance between dimensions is zero. However, if the shape of the lattice unit area is a hypercube, the covariance between dimensions can be made zero by rotating the coordinate axes of the lattice space so that they are parallel to each side of the hypercube. Therefore, formulas (1) and (2) are formulas that hold when the coordinate axes are rotated even if the covariance between dimensions is not zero. Note that, as is well known in linear algebra, the rotation of the coordinate axes is a unitary transformation, and is therefore not a transformation that changes the contents of formulas (1) and (2). Note that the process of making the covariance zero is a process of diagonalizing the matrix that represents the variance between dimensions.

In this way, the main data side probability estimation process estimates the occurrence probability of each element of the tensor representing the main data features using the result of integrating the parameterized probability density function with a lattice unit region whose shape is a hyper-rectangle as the integration region.

<Processing for determining the range of a hypercuboid>
So far, we have explained the use of a hyper-rectangle as a lattice unit region, but have not mentioned how to determine the shape and size of the hyper-rectangle. Here, we will explain an example of a process for determining the shape and size of a lattice unit region that at least satisfies the condition that it is a hyper-rectangle (hereinafter referred to as the "hyper-rectangle determination process").

In the hypercube determination process, first, the adjacent lattices of the origin lattice are calculated. The origin lattice is the lattice point located at the origin of the lattice space. An adjacent lattice is the lattice point next to the origin. Next, in the hypercube determination process, a hypercube that satisfies the hypercube determination conditions is calculated. The hypercube determination conditions include the condition that the lattice unit area of the origin lattice does not overlap with the lattice unit areas of each adjacent lattice, and the condition that the volume of the hypercube is equal to the volume of the Voronoi region.

As a specific example of the hypercuboid determination process, the conditions used to determine the shape and size of a hypercuboid in an 8-dimensional E8 lattice space will be described. There are two types of adjacent lattices of the origin lattice. One is [±1 ² , 0 ⁶ ]. The other is [±(1/2) ⁸ ]. Here, the superscript indicates the number of dimensions. For example, [1 ² , 0 ⁶ ] means [1, 1, 0, 0, 0, 0, 0, 0]. Note that the above brackets [ ] represent vectors.

<Notation for indicating a hypercuboid>
The superscripts in the hypercuboid [s ₁ ^a , s ₂ ^b . . . s ₃ ^c ] indicate that the length of each side from the first dimension to the a-th dimension is s ₁ , and the length of each side from the (a+1)th dimension to the (a+b)th dimension is s _2. In this way, the superscripts in the notation using superscripts indicate that the length of the sides of the hypercuboid is the same length s for a series of dimensions that are the same as the number of superscripts, with the character to which the superscript is attached being s. That is, s ^a indicates that the length of the sides of the hypercuboid is s for a series of a dimensions. Note that information on the order of the dimensions is required to determine whether the dimensions are continuous or not, but the order of the dimensions is a predetermined order.

Note that the "±" in [±1 ² , 0 ⁶ ] indicates that there are cases of +1 and -1. Therefore, [±1 ² , 0 ⁶ ] specifically means the four vectors [1, 1, 0 ⁶ ], [-1, 1, 0 ⁶ ], [1, -1, 0 ⁶ ], and [-1, -1, 0 ⁶ ].

Returning to the explanation of determining the shape and size of a hypercube in an eight-dimensional lattice space, there are two conditions for the hypercube of each adjacent lattice not to overlap with the hypercube of the origin lattice. The first is that seven or more of the sides (elements) of the hypercube are 1 or less. The other is that one or more of the sides (elements) of the hypercube are 1/2 or less.

Furthermore, since the volume of the Voronoi region of the E8 lattice is 1, the hypercuboid satisfies the following equation (4).

In the case of an eight-dimensional E8 lattice space, for example, s=[1/2, 1 ⁶ , 2] is obtained as a hypercuboid that satisfies these conditions.

Here is an example for other dimensions. In the case of a two-dimensional A2 lattice space, the range of the lattice unit area is the range enclosed by a hypercuboid expressed by the following equation (5).

In the case of 8-dimensional E8 lattice space, the range of the lattice unit area is the range enclosed by a hypercuboid expressed by the following equation (6).

In the case of a 24-dimensional Reach lattice space, the range of a lattice unit area is the range enclosed by a hypercuboid expressed by the following equation (7).

In this way, the extent of the hypercuboid (the shape and size of the lattice unit area) is determined. Returning to the explanation of Figure 1.

The main entropy acquisition unit 110 acquires the entropy of the main data features based on the estimation result of the main data side probability estimation unit 109. Hereinafter, the entropy of the main data features is referred to as the main entropy.

The main data side decoding unit 111 executes a main data feature decoding process. The main data feature decoding process is a process in which information obtained based on the main data feature is processed, and is a process in which the process object is decoded. The process object of the main data feature decoding process executed by the main data side decoding unit 111 is the noise-added main data feature. The contents of the decoding process of the main data side decoding unit 111 are updated by learning.

The reconstruction error calculation unit 112 calculates the difference between the decoding result of the main data side decoding unit 111 and the main data acquired by the main data acquisition unit 101. Hereinafter, the difference between the decoding result of the main data side decoding unit 111 and the main data acquired by the main data acquisition unit 101 is referred to as the reconstruction error. The difference between the decoding result of the main data side decoding unit 111 and the main data acquired by the main data acquisition unit 101 may be expressed, for example, by the mean squared error sum or by binary cross entropy.

The optimization unit 113 updates the learning network 100 based on the auxiliary entropy, the principal entropy, and the reconstruction error. The auxiliary entropy, the principal entropy, and the reconstruction error are all examples of the output of the learning network 100. Specifically, the optimization unit 113 updates the learning network 100 so as to reduce the optimization error, the principal entropy, and the auxiliary entropy. The objective function used by the optimization unit 113 is, for example, L=D+λ(R _y +R _z ).

The symbol L represents the objective function. The symbol D represents the reconstruction error. The symbol lambda is a predefined constant. The symbol R _y represents the auxiliary entropy. The symbol R _z represents the primal entropy.

Since a small entropy means a short code length, the optimization unit 113 updates the learning network 100 to reduce the entropy. Also, since a smaller optimization error means higher accuracy of self-encoding, the optimization unit 113 updates the learning network 100 to reduce the optimization error.

For example, the learning network 100 is updated by using a gradient method to solve the minimization problem of the objective function L. That is, the learning network 100 is updated by updating the values of each parameter of the learning network 100 using, for example, the backpropagation method.

Specifically, updating the learning network 100 means updating the processing contents of the main data side encoding unit 102, the auxiliary data side encoding unit 103, the auxiliary data side probability estimation unit 106, the auxiliary data side decoding unit 107, and the main data side decoding unit 111.

<Relationship between autoencoding and learning network 100>
The learning network 100 includes a main data-side noise adding unit 104. The main data-side noise adding unit 104 itself does not perform quantization processing. However, since the processing of the main data-side noise adding unit 104 is included, the performance of self-encoding using vector quantization is improved by performing learning. The reason for this will be explained below.

For example, as described in Reference 1 below, when vector quantization is performed during learning, the gradient becomes 0, making it known that learning that makes the self-encoding process using vector quantization highly efficient cannot be performed. Therefore, in order to make the self-encoding process more efficient through learning, a process of adding noise to the vectors is performed during learning instead of vector quantization itself.

By performing a process of adding noise instead of vector quantization itself, the performance of other processes that generate the information used in vector quantization is improved, rather than the process of performing vector quantization itself. As a result, even if vector quantization is performed instead of a process of adding noise during self-encoding, self-encoding is more efficient than before learning.

Reference 1: Balle, "Variational image compression with a scale hyperprior," 2018

Regarding the learning unit 10, the main data side noise adding unit 104 adds noise to the noise-added vector. Therefore, by performing learning including the main data side noise adding unit 104, the gradient does not become 0. As a result, it is possible to update the contents of the main data side encoding unit 102, the auxiliary data side encoding unit 103, the auxiliary data side probability estimating unit 106, the auxiliary data side decoding unit 107, and the main data side decoding unit 111, which are also used in self-encoding using vector quantization.

Therefore, what has been described so far as the network learning result specifically means the peripheral processing updated by the learning of the neural network (i.e., learning network 100) that adds noise instead of vector quantization. The peripheral processing is processing that generates information used in vector quantization. Specifically, the peripheral processing is processing that is executed by each of the main data side encoding unit 102, the auxiliary data side encoding unit 103, the auxiliary data side probability estimating unit 106, the auxiliary data side decoding unit 107, and the main data side decoding unit 111.

7 is a diagram showing an example of a flow of processing executed by the learning unit 10 in the embodiment. The main data acquisition unit 101 acquires main data x=[ _x1 , _x2 , ..., xN] (step S101). The main data x is an N-dimensional vector having N elements from _x1 to _xN . Each element from _x1 to _xN is a tensor. Therefore, each element from _x1 to _xN may be a scalar or a _vector .

Next, the main data side encoding unit 102 executes a main data feature acquisition process (step S102). That is, the main data side encoding unit 102 encodes the main data x. By encoding the main data, a main data feature y=f _enc (x) is obtained. The function f _enc (x) is a function (hereinafter referred to as a "main data encoding function") that expresses the encoding process of the main data x. Note that the main data feature y is a tensor composed of k K-dimensional vectors.

Next, the auxiliary data side encoding unit 103 executes an auxiliary feature acquisition process (step S103). By executing the auxiliary feature acquisition process, an auxiliary feature z = g _enc (y) is obtained. The function g _enc (y) is a function expressing the process of encoding the main data feature y (hereinafter referred to as the "main data feature encoding function"). Note that the auxiliary feature z is a tensor such as a vector.

Next, the main data-side noise adding unit 104 executes a noise-added main data feature acquisition process (step S104). By executing the noise-added main data feature acquisition process, noise is added to the main data feature. That is, by executing the noise-added main data feature acquisition process, the noise-added main data feature y^=[ _y1 ^, _y2 ^, ..., _yk ^] is obtained, where _yi ^= _yi + _uy . _yi represents the i-th vector element of the main data feature y. _uy represents noise.

Next, the auxiliary data-side noise adding unit 105 executes a noise-added auxiliary feature acquisition process (step S105). By executing the noise-added auxiliary feature acquisition process, noise is added to the auxiliary feature. That is, by executing the noise-added auxiliary feature acquisition process, noise-added auxiliary feature z^=[z ₁ ^, z ₂ ^, ..., z _w ^] is obtained. Note that w is an integer equal to or greater than 1. However, z _i ^=z _i +u _z . _{Z i} represents the i-th element of the auxiliary feature z. _{U z} represents noise.

Next, the auxiliary data side probability estimation unit 106 executes the auxiliary data side probability estimation process (step S106). The auxiliary data side probability estimation unit 106 estimates the auxiliary data side probability by executing the auxiliary data side probability estimation process. Specifically, the auxiliary data side probability is expressed by the following formula (8).

The symbol on the left side of equation (8) represents the probability of the auxiliary data. The symbol h is the cumulative distribution function of the parametrized auxiliary feature.

Next, the auxiliary data side decoding unit 107 executes auxiliary feature decoding processing on the noise-added auxiliary feature (step S107). By executing the auxiliary feature decoding processing, the noise-added auxiliary feature is decoded. That is, by executing the auxiliary feature decoding processing, auxiliary data θ=g _dec (z^) is obtained. The function g _dec (z^) is a function (hereinafter referred to as the "auxiliary feature decoding function") that expresses the process of decoding the noise-added auxiliary feature z^. Note that the auxiliary data θ is a tensor such as a vector.

Next, the auxiliary entropy acquisition unit 108 acquires the auxiliary entropy based on the auxiliary data side probability (step S108). Specifically, the auxiliary entropy acquisition unit 108 acquires the auxiliary entropy by executing the process represented by the following formula (9).

The symbol on the left side of equation (9) represents auxiliary entropy.

Next, the main data side probability estimation unit 109 executes a main data side probability estimation process (step S109). The main data side probability estimation unit 109 executes the main data side probability estimation process to estimate the main data side probability based on the noise-added main data features and the auxiliary data. The main data side probability is specifically expressed by the above-mentioned formula (1).

Next, the principal entropy acquisition unit 110 acquires the principal entropy based on the main data side probability (step S110). Specifically, the principal entropy acquisition unit 110 acquires the principal entropy by executing the process represented by the following formula (10).

The symbol on the left side of equation (10) represents the principal entropy.

Next, the main data side decoding unit 111 executes a main data feature decoding process on the noise-added main data feature (step S111). The noise-added main data feature is decoded by executing the main data feature decoding process. Hereinafter, information obtained by decoded the noise-added main data feature is referred to as decoded main data. Therefore, the main data side decoding unit 111 obtains decoded main data x^=f _dec (y^) by executing the main data feature decoding process. The function f _dec (y^) is a function (hereinafter referred to as the "main data feature decoding function") that expresses the process of decoding the noise-added main data feature y^. Note that the decoded main data x^ is a tensor such as a vector.

Next, the reconstruction error calculation unit 112 acquires the difference between the decoded main data and the main data acquired by the main data acquisition unit 101 (step S112). The difference between the decoded main data and the main data acquired by the main data acquisition unit 101 is the reconstruction error.

Next, the optimization unit 113 updates the learning network 100 based on the auxiliary data entropy, the main data entropy, and the reconstruction error (step S113). Next, the optimization unit 113 determines whether a predetermined end condition for learning (hereinafter referred to as the "learning end condition") is satisfied (step S114). The learning end condition is, for example, a condition that the learning network 100 has been updated a predetermined number of times.

If the learning end condition is met (step S114: YES), the process ends. On the other hand, if the learning end condition is not met (step S114: NO), the process returns to step S101. The peripheral process at the time when the learning end condition is met is used for vector quantization as the learned peripheral process.

Note that the processes in steps S101 to S114 may be performed in any order as long as they do not violate the law of causality.

As described above, the auxiliary data side probability estimation unit 106 may obtain a predetermined probability distribution by reading it from a specified storage device, etc. In such a case, the contents of the auxiliary data side probability estimation process are not updated by learning. Therefore, the learned auxiliary data side probability estimation process is the same as the auxiliary data side probability estimation process before learning.

More specifically, the update of the contents of the auxiliary data side probability estimation process is an update of the parametrized auxiliary feature cumulative distribution function h. Therefore, when the auxiliary data side probability estimation unit 106 acquires a pre-given probability distribution by reading from a predetermined storage device, the learned parametrized auxiliary feature cumulative distribution function h is the same as the parametrized auxiliary feature cumulative distribution function h before learning.

In this way, the learning unit 10 updates the encoding and decoding processes in the self-encoding process using vector quantization through learning. The self-encoding process using vector quantization uses main data features, which are features to be self-encoded, and auxiliary features, which are features of the main data features. Furthermore, the self-encoding process using vector quantization performs entropy coding on the result of vector quantizing the main data features, and entropy coding on the result of scalar quantizing the auxiliary features.

The encoding process in such a self-encoding process using vector quantization specifically includes a main data feature acquisition process performed by the main data side encoding unit 102 and an auxiliary feature acquisition process performed by the auxiliary data side encoding unit 103. The decoding process in such a self-encoding process using vector quantization specifically includes an auxiliary feature decoding process performed by the auxiliary data side decoding unit 107 and a main data feature decoding process performed by the main data side decoding unit 111.

Self-encoding using vector quantization using learned peripheral processing will be explained using Figures 8 and 9. "Learned" means at the point in time when the learning end condition is met. More specifically, as an example of a device that performs self-encoding using vector quantization using learned peripheral processing, a self-encoding device 2 that performs encoding and decoding processes will be explained. The self-encoding device 2 is a type of autoencoder (autoencoder).

Figure 8 is a first explanatory diagram illustrating an overview of the self-encoding device 2 in an embodiment. Figure 9 is a second explanatory diagram illustrating an overview of the self-encoding device 2 in an embodiment. More specifically, Figure 8 is an explanatory diagram illustrating the encoding process performed by the self-encoding device 2, and Figure 9 is an explanatory diagram illustrating the decoding process performed by the self-encoding device 2.

Since the self-encoding device 2 is a type of self-encoder, it includes an encoder and a decoder. Specifically, the self-encoding device 2 includes an encoder 200 and a decoder 212. The encoder 200 includes a self-encoding target acquisition unit 201, a learned main data side encoding unit 202, a learned auxiliary data side encoding unit 203, a vector quantization unit 204, a scalar quantization unit 205, a learned auxiliary data side probability estimation unit 206, a learned auxiliary data side decoding unit 207, an auxiliary entropy encoding unit 208, a main data side probability estimation unit 209, a main entropy encoding unit 210, and a data multiplexing unit 211.

The decoder 212 includes an encoded data acquisition unit 213, a data separation unit 214, an auxiliary entropy decoding unit 215, a learned auxiliary data side decoding unit 216, a main entropy decoding unit 217 and a learned main data side decoding unit 218.

The self-encoding target acquisition unit 201 acquires data that is the target of self-encoding as main data. Hereinafter, the target of self-encoding is referred to as the self-encoding target.

The learned main data side encoding unit 202 executes a learned main data feature acquisition process for the self-encoding target. By executing the learned main data feature acquisition process, the learned main data side encoding unit 202 acquires the main data feature of the self-encoding target.

The learned auxiliary data side encoding unit 203 executes a learned auxiliary feature acquisition process on the main data features to be self-encoded. By executing the learned auxiliary feature acquisition process, the learned auxiliary data side encoding unit 203 acquires the auxiliary features to be self-encoded.

The vector quantization unit 204 performs a vector quantization process on the main data features to be self-encoded. By performing the vector quantization process, the vector quantization unit 204 obtains vector quantized main data features (hereinafter referred to as "vector quantized features") to be self-encoded.

The scalar quantization unit 205 performs a scalar quantization process on the auxiliary features to be self-encoded. By performing the scalar quantization process, the scalar quantization unit 205 obtains scalar quantized auxiliary features (hereinafter referred to as "scalar quantized features") of the self-encoding target.

The learned auxiliary data side probability estimation unit 206 executes a learned auxiliary data side probability estimation process. The learned auxiliary data side probability estimation unit 206 executes a learned auxiliary data side probability estimation process to estimate the auxiliary data side probability of the self-encoding target based on the scalar quantized feature.

The learned auxiliary data side decoding unit 207 executes a learned auxiliary feature decoding process on the scalar quantized feature. That is, the learned auxiliary data side decoding unit 207 decodes the scalar quantized feature. Hereinafter, information obtained by decoded the scalar quantized feature is referred to as quantized auxiliary data. Therefore, the learned auxiliary data side decoding unit 207 is a process for obtaining quantized auxiliary data by decoding the scalar quantized feature.

The auxiliary entropy coding unit 208 performs entropy coding of the scalar quantized feature based on the scalar quantized feature and the probability of the auxiliary data to be self-encoded. The entropy coding is, for example, arithmetic coding.

The main data side probability estimation unit 209 estimates the main data side probability of the self-encoding target based on the vector quantization feature and the quantization auxiliary data.

The main entropy coding unit 210 performs entropy coding of the vector quantization feature based on the vector quantization feature and the main data side probability of the self-encoding target. The entropy coding is, for example, arithmetic coding.

The data multiplexing unit 211 outputs the entropy-coded vector quantization features and the entropy-coded scalar quantization features to the decoder 212. In this manner, the encoder 200 encodes the self-encoding target.

The encoded data acquisition unit 213 acquires entropy-encoded vector quantization features and entropy-encoded scalar quantization features.

The data separation unit 214 acquires the entropy-coded vector quantization features and the entropy-coded scalar quantization features acquired by the encoded data acquisition unit 213. The data separation unit 214 outputs the entropy-coded scalar quantization features to the auxiliary entropy decoding unit 215, and outputs the entropy-coded vector quantization features to the main entropy decoding unit 217.

The learned auxiliary entropy decoding unit 215 performs entropy decoding on the entropy-encoded scalar quantized features using a learned parametrized auxiliary feature cumulative distribution function.

The learned auxiliary data side decoding unit 216 performs learned auxiliary feature decoding processing on the result of entropy decoding by the learned auxiliary entropy decoding unit 215.

The main entropy decoding unit 217 performs entropy coding of the entropy coded vector quantization feature based on the entropy coded vector quantization feature and the result of the learned auxiliary feature decoding process by the learned auxiliary data side decoding unit 216. More specifically, the main entropy decoding unit 217 performs entropy decoding of the entropy coded vector quantization feature using a decoding cumulative distribution function. The decoding cumulative distribution function is a parameterized main data feature cumulative distribution function whose parameter value is the value indicated by the result of the learned auxiliary feature decoding process by the learned auxiliary data side decoding unit 216.

The learned main data side decoding unit 218 performs a learned main data feature decoding process on the result of decoding by the main entropy decoding unit 217.

In this way, the decoder 212 decodes the self-encoding target encoded by the encoder 200. Also, in this way, the self-encoding device 2 self-encodes the self-encoding target.

10 is a flowchart showing an example of the flow of processing executed by the encoder 200 in the embodiment. The self-encoding target acquisition unit 201 acquires the self-encoding target X=[X ₁ , X ₂ , ..., X _N ] (step S201). The self-encoding target X is an N-dimensional vector having N elements from X ₁ to X _N. Each element from X ₁ to X _N is a tensor. Therefore, each element from X ₁ to X _N may be a scalar or a vector.

Next, the learned main data side encoding unit 202 executes a learned main data feature acquisition process (step S202). That is, the learned main data side encoding unit 202 encodes the auto-encoding target X. By encoding the auto-encoding target, a main data feature Y of the auto-encoding target = F _enc (Y) is obtained. The function F _enc (X) is a learned main data encoding function. Note that the main data feature Y is a tensor such as a vector.

Next, the learned auxiliary data side encoding unit 203 executes a learned auxiliary feature acquisition process (step S203). By executing the learned auxiliary feature acquisition process, an auxiliary feature Z to be self-encoded = G _enc (Y) is obtained. The function G _enc (Y) is a learned main data feature encoding function. Note that the auxiliary feature Z is a tensor such as a vector.

Next, the vector quantization unit 204 performs vector quantization on the main data feature Y to be self-encoded (step S204). By performing vector quantization, the vector quantized feature Y^=[ _Y1 ^, _Y2 ^, ..., _Yk ^]=[Q( _Y1 ), Q( _Y2 ), ..., Q( _Yk )] is obtained. _Yi represents the i-th element of the main data feature Y to be self-encoded. Y^ _i represents the i-th element of the vector quantized feature Y^. Q is a function expressed by the following formula (11).

Note that the symbol Λ represents the set of all lattice points.

Next, the scalar quantization unit 205 performs scalar quantization on the auxiliary feature Z to be self-encoded (step S205). By executing the scalar quantization process, the scalar quantization unit 205 obtains the scalar quantized feature Z^ = round(Z). Note that round represents a rounding process.

Next, the learned auxiliary data side probability estimation unit 206 executes a learned auxiliary data side probability estimation process (step S206). The learned auxiliary data side probability estimation unit 206 executes the learned auxiliary data side probability estimation process to estimate the auxiliary data side probability of the self-encoding target based on the scalar quantized feature Z^. Specifically, the auxiliary data side probability of the self-encoding target is expressed by the following formula (12).

The symbol on the left side of equation (12) represents the probability of the auxiliary data to be auto-encoded. The symbol H is the cumulative distribution function of the learned parametrized auxiliary features.

The learned auxiliary data decoding unit 207 executes a learned auxiliary feature decoding process on the scalar quantized feature Z^ (step S207). That is, the learned auxiliary data decoding unit 207 obtains quantized auxiliary data Θ=G _dec (Z^) by decoding the scalar quantized feature. The function G _dec (Z^) is a learned auxiliary feature decoding function. Note that the quantized auxiliary data Θ is a tensor such as a vector.

Next, the auxiliary entropy coding unit 208 performs entropy coding of the scalar quantized feature Z^ based on the scalar quantized feature Z^ and the probability of the auxiliary data to be self-encoded (step S208).

Next, the main data side probability estimation unit 209 estimates the main data side probability of the self-encoding target based on the vector quantization feature Y^ and the quantization auxiliary data Θ (step S209).

Next, the main entropy coding unit 210 performs entropy coding of the vector quantized feature Y^ based on the vector quantized feature Y^ and the main data side probability to be self-encoded (step S210).

Next, the data multiplexing unit 211 outputs the entropy-coded vector quantization features and the entropy-coded scalar quantization features to the decoder 212 (step S211).

The series of processes from step S201 to step S211 is an example of the encoding process by the encoder 200. Note that the processes from step S201 to step S211 may be executed in any order as long as they do not violate the law of causality.

Figure 11 is a flowchart showing an example of the flow of processing executed by the decoder 212 in an embodiment. The encoded data acquisition unit 213 acquires the result of encoding by the encoder 200 (step S301). Specifically, the result of encoding by the encoder 200 is the entropy-encoded vector quantization feature and the entropy-encoded scalar quantization feature output in step S211.

Next, the data separation unit 214 separates the entropy-coded scalar quantization feature acquired in step S301 from the entropy-coded vector quantization feature acquired in step S302 (step S302). Specifically, separating means outputting the entropy-coded scalar quantization feature acquired in step S301 to the auxiliary entropy decoding unit 215, and outputting the entropy-coded vector quantization feature acquired in step S302 to the main entropy decoding unit 217.

Next, the learned auxiliary entropy decoding unit 215 performs entropy decoding on the entropy-encoded scalar quantized features using the learned parametrized auxiliary feature cumulative distribution function (step S303).

Next, the learned auxiliary data side decoding unit 216 executes a learned auxiliary feature decoding process on the result of entropy decoding by the learned auxiliary entropy decoding unit 215 (step S304).

Next, the principal entropy decoding unit 217 performs entropy decoding of the entropy-encoded vector quantization features using the decoded cumulative distribution function (step S305).

Next, the learned main data side decoding unit 218 executes a learned main data feature decoding process on the result of decoding by the main entropy decoding unit 217 (step S306).

The series of processes from step S301 to step S306 is an example of the decoding process by the decoder 212. Note that each process from step S301 to step S306 is executed after the encoding process by the encoder 200 such as step S211, and may be executed in any order as long as it does not violate the law of causality.

<Hardware Description>
12 is a diagram showing an example of a hardware configuration of the learning device 1 in the embodiment. The learning device 1 includes a control unit 11 including a processor 91 such as a CPU (Central Processing Unit) and a memory 92 connected by a bus, and executes a program. The learning device 1 functions as a device including the control unit 11, an input unit 12, a communication unit 13, a storage unit 14, and an output unit 15 by executing the program.

More specifically, the processor 91 reads out a program stored in the storage unit 14 and stores the read out program in the memory 92. The processor 91 executes the program stored in the memory 92, whereby the learning device 1 functions as a device including a control unit 11, an input unit 12, a communication unit 13, a storage unit 14 and an output unit 15.

The control unit 11 controls the operation of various functional units of the learning device 1. The control unit 11 controls, for example, the operation of the output unit 15. The control unit 11 records, for example, various pieces of information generated by learning in the memory unit 14.

The input unit 12 includes input devices such as a mouse, a keyboard, and a touch panel. The input unit 12 may be configured as an interface that connects these input devices to the learning device 1. The input unit 12 accepts input of various information for the learning device 1.

The communication unit 13 includes a communication interface for connecting the learning device 1 to an external device. The communication unit 13 communicates with the external device via wired or wireless communication. The external device is, for example, a device that transmits main data used for learning. The communication unit 13 acquires the main data used for learning by communicating with the device that transmits the main data. The external device is, for example, the self-encoding device 2. The communication unit 13 transmits the network learning results to the self-encoding device 2 by communicating with the self-encoding device 2. Note that the main data does not necessarily need to be input via the communication unit 13, and may be input to the input unit 12.

The memory unit 14 is configured using a computer-readable storage medium device such as a magnetic hard disk device or a semiconductor memory device. The memory unit 14 stores various information related to the learning device 1. The memory unit 14 stores information inputted, for example, via the input unit 12 or the communication unit 13. The memory unit 14 stores various information generated, for example, by the execution of learning.

The memory unit 14 pre-stores, for example, a probability distribution used to obtain the occurrence probability of each element of a tensor indicating an auxiliary feature. The memory unit 14 pre-stores, for example, a parametrized auxiliary feature cumulative distribution function. The memory unit 14 pre-stores, for example, a parametrized main data feature cumulative distribution function. The memory unit 14 pre-stores, for example, representative vector information. The memory unit 14 stores, for example, the result of hypercuboid division.

The memory unit 14 stores, for example, initial values of each parameter of the learning network 100 in advance. The initial values are, for example, random values. The memory unit 14 stores, for example, the network learning results.

The output unit 15 outputs various types of information. The output unit 15 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 15 may be configured as an interface that connects these display devices to the learning device 1. The output unit 15 outputs information input to the input unit 12, for example. The output unit 15 may display the results of learning, for example.

Figure 13 is a diagram showing an example of the configuration of the control unit 11 provided in the learning device 1 in an embodiment. The control unit 11 includes a learning unit 10, a memory control unit 120, a communication control unit 130, and an output control unit 140. The memory control unit 120 records various information in the memory unit 14. The communication control unit 130 controls the operation of the communication unit 13. The output control unit 140 controls the operation of the output unit 15.

Figure 14 is a diagram showing an example of the hardware configuration of the self-encoding device 2 in an embodiment. The self-encoding device 2 has a control unit 21 including a processor 93 such as a CPU (Central Processing Unit) and a memory 94 connected by a bus, and executes a program. The self-encoding device 2 functions as a device including the control unit 21, input unit 22, communication unit 23, memory unit 24, and output unit 25 by executing the program.

More specifically, the processor 93 reads out the program stored in the storage unit 24 and stores the read out program in the memory 94. The processor 93 executes the program stored in the memory 94, whereby the self-encoding device 2 functions as a device including a control unit 21, an input unit 22, a communication unit 23, a storage unit 24 and an output unit 25.

The control unit 21 controls the operation of various functional units of the self-encoding device 2. The control unit 21 controls, for example, the operation of the output unit 25. The control unit 21 records, for example, various pieces of information generated by encoding by the encoder 200 and decoding by the decoder 212 in the memory unit 24.

The input unit 22 includes input devices such as a mouse, a keyboard, and a touch panel. The input unit 22 may be configured as an interface that connects these input devices to the self-encoding device 2. The input unit 22 accepts input of various information for the self-encoding device 2.

The communication unit 23 includes a communication interface for connecting the self-encoding device 2 to an external device. The communication unit 23 communicates with the external device via wired or wireless communication. The external device is, for example, a device that transmits the self-encoding target. The communication unit 23 acquires the self-encoding target by communicating with the device that transmits the self-encoding target. The external device is, for example, the learning device 1. The communication unit 23 receives the network learning result by communicating with the learning device 1. Note that the self-encoding target does not necessarily have to be input via the communication unit 23, and may be input to the input unit 22.

The memory unit 24 is configured using a computer-readable storage medium device such as a magnetic hard disk device or a semiconductor storage device. The memory unit 24 stores various information related to the self-encoding device 2. The memory unit 24 stores information inputted, for example, via the input unit 22 or the communication unit 23. The memory unit 24 stores various information generated, for example, by the execution of encoding by the encoder 200 and decoding by the decoder 212.

The memory unit 24 stores, for example, the network learning results. The memory unit 24 stores, for example, representative vector information in advance. The memory unit 24 stores, for example, the results of hypercuboid division.

The output unit 25 outputs various information. The output unit 25 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 25 may be configured as an interface that connects these display devices to the learning device 1. The output unit 25 outputs, for example, information input to the input unit 22. The output unit 25 may output, for example, the result of auto-encoding of the auto-encoding target.

Figure 15 is a diagram showing an example of the configuration of the control unit 21 provided in the self-encoding device 2 in an embodiment. The control unit 21 includes a self-encoding execution unit 20, a memory control unit 220, a communication control unit 230, and an output control unit 240. The self-encoding execution unit 20 performs self-encoding on the self-encoding target. The self-encoding execution unit 20 includes an encoder 200 and a decoder 212. The self-encoding execution unit 20 performs self-encoding of the self-encoding target by encoding using the encoder 200 and decoding using the decoder 212.

The memory control unit 220 records various information in the memory unit 24. The communication control unit 230 controls the operation of the communication unit 23. The output control unit 240 controls the operation of the output unit 25.

The learning device 1 configured in this manner learns peripheral processing for vector quantization using representative vector information, which is information indicating the positions of representative vectors arranged in a lattice pattern in a vector space such as LatticeVQ. The learning device 1 then estimates the occurrence probability of the representative vector using the results of the hypercuboid division. As described above, when using representative vectors, the occurrence probability of the representative vector may be estimated using Voronoi division, but integration is not easy.

Therefore, the learning device 1, which estimates the occurrence probability of a representative vector using a hypercuboid obtained by hypercuboid division, can reduce the burden required to obtain the process of self-encoding using vector quantization. This reduces the burden from the learning stage required to realize self-encoding using vector quantization. Therefore, the learning device 1 can reduce the burden required for self-encoding using vector quantization.

Furthermore, the learning device 1 configured in this manner uses the representative vector to learn peripheral processing for vector quantization, so the burden required for learning the representative vector can be reduced. Furthermore, the learning device 1 configured in this manner uses the representative vector to learn peripheral processing for vector quantization, so there is no need to use memory, which is required when learning the representative vector. Therefore, the learning device 1 can reduce the frequency with which problems of insufficient memory occur, and can process main data of larger dimensions.

As mentioned above, in learning vector quantization, it is necessary to add noise instead of quantization. However, when using representative vector information such as LatticeVQ, the range of the Voronoi region becomes a quantization error, so it is not easy to generate noise whose probability distribution is Gaussian.

Meanwhile, the learning device 1 configured in this manner uses, in learning, as noise, only samples within the Voronoi region among samples randomly generated within a (K-1)-dimensional sphere circumscribing a Voronoi region in a K-dimensional vector space. This makes it possible to generate noise that follows a Gaussian distribution. Therefore, the learning device 1 can reduce the burden required to obtain the process of self-encoding using vector quantization. Therefore, the learning device 1 can reduce the burden required for self-encoding using vector quantization.

The self-encoding device 2 configured in this manner performs self-encoding using vector quantization using the results of learning by the learning device 1. This reduces the burden of self-encoding using vector quantization.

(Modification)
In adding noise to the main data feature quantity, samples uniformly generated within a (K-1)-dimensional sphere whose volume is approximated to the volume of a Voronoi region in a K-dimensional vector space may be used as noise points. In adding noise to the main data feature quantity, samples uniformly generated within a hypercuboid obtained by hypercuboid division may be used as noise points.

Therefore, the process of adding noise to the main data features (i.e., vector noise adding process) may be any one of the first noise adding process, the second noise adding process, or the third noise adding process. The first noise adding process is a process of adding, as noise, samples within a Voronoi region among samples randomly generated within a (K-1)-dimensional sphere circumscribing a Voronoi region in a K-dimensional vector space. In other words, the first noise adding process is the process described using Figures 3 and 4.

The second noise-adding process is a process of adding, as noise, samples that are uniformly generated within a (K-1)-dimensional sphere whose volume is approximating the volume of a Voronoi region in a K-dimensional vector space. The third noise-adding process is a process of adding, as noise, samples that are uniformly generated within a region that divides a vector space in which representative vectors are arranged in a lattice pattern, that includes one lattice point of the vector space, and that has a shape of a hypercuboid.

The learning device 1 may be implemented using multiple information processing devices connected to each other via a network. In this case, each functional unit of the learning device 1 may be distributed and implemented in multiple information processing devices.

The self-encoding device 2 may be implemented using a plurality of information processing devices communicably connected via a network. In this case, each functional unit of the self-encoding device 2 may be distributed and implemented in a plurality of information processing devices.

All or part of the functions of the learning device 1 and the self-encoding device 2 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

The above describes in detail an embodiment of the present invention with reference to the drawings, but the specific configuration is not limited to this embodiment and also includes designs that do not deviate from the gist of the present invention.

1...Learning device, 10...Learning unit, 100...Learning network, 101...Main data acquisition unit, 102...Main data side encoding unit, 103...Auxiliary data side encoding unit, 104...Main data side noise adding unit, 105...Auxiliary data side noise adding unit, 106...Auxiliary data side probability estimation unit, 107...Auxiliary data side decoding unit, 108...Auxiliary entropy acquisition unit, 109...Main data side probability estimation unit, 110...Main entropy acquisition unit, 111...Main data side decoding unit, 112...Reconstruction error calculation unit, 113...Optimization unit, 2...Self-encoding device, 200...Encoder, 201...Self-encoding target acquisition unit, 202...Learned main data side encoding unit, 203...Learned auxiliary data side encoding unit, 204...Vector quantization unit, 205...Scalar quantization unit, 206...Learned auxiliary data side probability estimation unit, 207...Learned auxiliary data side decoding unit, 208...Auxiliary entropy coding unit, 209...Main data side probability estimation unit, 210...Main entropy coding unit, 211...Data multiplexing unit, 213...Encoded data acquisition unit, 214...Data separation unit, 215...Auxiliary entropy decoding unit, 216...Learned auxiliary data side decoding unit, 217...Main entropy decoding unit, 218...Learned main data side decoding unit, 11...Control unit, 12...Input unit, 13...Communication unit, 14...Memory unit, 15...Output unit, 120...Memory control unit, 130...Communication control unit, 140...Output control unit, 21...Control unit, 22...Input unit, 23...Communication unit, 24...Memory unit, 25...Output unit, 20...Self-encoding execution unit, 220: storage control unit, 230: communication control unit, 240: output control unit, 91: processor, 92: memory, 93: processor, 94: memory

Claims (8)

a learning unit that updates, by learning, encoding and decoding processes in the auto-encoding process using vector quantization, the auto-encoding process using main data features that are features to be auto-encoded and auxiliary features that are features of the main data features, and that performs entropy coding on a result of vector quantizing the main data features and entropy coding on a result of scalar quantizing the auxiliary features;
Equipped with
the learning unit executes a main data side probability estimation process in the learning to estimate an occurrence probability of each element of a tensor indicating the main data feature amount;
The main data-side probability estimation process estimates the occurrence probability using a result of integrating a probability density function that is parameterized using an integration region that divides a vector space in which representative vectors are arranged in a lattice pattern, includes one lattice point of the vector space, and has a shape of a hypercuboid as an integration region.
Learning device. the learning unit updates the encoding and decoding processes using an entropy of the main data feature amount obtained based on the occurrence probability estimated by the main data side probability estimation process.
The learning device according to claim 1 . The vector quantization is LatticeVQ.
The learning device according to claim 1 or 2. an auto-encoding target acquisition unit that acquires an auto-encoding target;
the auto-encoding process using vector quantization is an auto-encoding process using main data features that are features to be auto-encoded and auxiliary features that are features of the main data features, the auto-encoding process performing entropy coding of a result of vector quantizing the main data features and entropy coding of a result of scalar quantizing the auxiliary features, wherein the learning unit performs a main data side probability estimation process in the learning to estimate an occurrence probability of each element of a tensor indicating the main data features, and the main data side probability estimation process estimates the occurrence probability using a result of integrating a probability density function that is parametrized using a region that divides a vector space in which representative vectors are arranged in a lattice, the region including one lattice point of the vector space, and the region having a shape of a hypercuboid, as an integral region; and an auto-encoding execution unit that performs auto-encoding by vector quantization of the target acquired by the auto-encoding target acquisition unit, using a learned encoding process and a learned decoding process obtained using a learning device;
A self-encoding device comprising: a learning step of updating, by learning, encoding and decoding processes in the auto-encoding process using vector quantization, the auto-encoding process using main data features which are features to be auto-encoded, and auxiliary features which are features of the main data features, and performing entropy coding on a result of vector quantizing the main data features and entropy coding on a result of scalar quantizing the auxiliary features;
having
the learning step executes a main data side probability estimation process for estimating occurrence probability of each element of a tensor indicating the main data feature amount in the learning;
The main data-side probability estimation process estimates the occurrence probability using a result of integrating a probability density function that is parameterized using an integration region that divides a vector space in which representative vectors are arranged in a lattice pattern, includes one lattice point of the vector space, and has a shape of a hypercuboid as an integration region.
How to learn. An auto-encoding target acquisition step of acquiring an auto-encoding target;
the self-encoding process using vector quantization is an auto-encoding process using main data features that are features to be auto-encoded and auxiliary features that are features of the main data features, the self-encoding process performing entropy coding on a result of vector quantizing the main data features and entropy coding on a result of scalar quantizing the auxiliary features, wherein the learning unit performs a main data side probability estimation process in the learning to estimate an occurrence probability of each element of a tensor indicating the main data features, and the main data side probability estimation process estimates the occurrence probability using a result of integrating a probability density function that is parametrized using a region that divides a vector space in which representative vectors are arranged in a lattice, the region including one lattice point of the vector space, and the region having a shape of a hypercuboid, as an integral region; and an auto-encoding execution step of performing auto-encoding by vector quantization of the object acquired in the auto-encoding target acquisition step using the learned encoding process and the learned decoding process obtained using a learning device;
The self-encoding method has the following structure: A program for causing a computer to function as a learning device according to any one of claims 1 to 3. A program for causing a computer to function as the self-encoding device described in claim 4.

Images