JP4611938B2 - Subjective quality estimation formula deriving method, subjective quality estimation formula deriving apparatus and processing program therefor - Google Patents

Description

The present invention relates to an image encoding / decoding technique, in particular, derive an expression for estimating the subjective quality of the user of the image relates to a technology that is available to determine the encoding parameters and the decoding parameters.

  In order to improve the quality of communication services using video media, it is necessary to construct a technique for estimating the subjective quality of a user for an image.

  Against this background, in the prior art, a method for estimating the “Mean Opinion Score (MOS)” using the bit rate and the image feature amount has been studied (for example, non-patent) References 1 and 2).

  In Non-Patent Documents 1 and 2, the average opinion score (MOS) obtained by many subjective evaluation experiments is approximated by a function including a small number of parameters, and the parameters of the function are estimated using the feature values obtained from the original image. The invention describes that an optimum frame rate is obtained by giving a specified bit rate to the estimated function and obtaining its maximum point.

As another conventional technique, the following Patent Document 1 describes video bit rate, frame rate, video invalid packet rate, audio invalid packet rate, delay as factors of quality degradation in the bidirectional communication service between terminals. The time is monitored, and based on these quality deterioration factors, an aesthetic factor score S1 and a dynamic factor score S2 are estimated as psychological factors for the quality received by the user, and the estimated factor scores S1, The invention of calculating an overall quality evaluation value MOS based on S2 is described. In this invention, a calculation formula for obtaining these factor scores S1 and S2 is obtained based on a subjective evaluation experiment. I try to derive.
Inazumi, Yoshida, Sakai, Hotta, "Optimal Frame Rate Estimation for Video Communication under Bitrate Restriction", IEICE Theory (B), vol.J85-B, no.7, pp.1130-1142, Jul .2002 Miyata, Yoshida, "Subjective image quality estimation based on SNR", IEICE theory (A), vol.J88-A, no.11, pp.1292-1296, Nov.2005 JP 2006-74333 A

  However, the conventional technique has a problem in that it cannot estimate the subjective quality of the image according to the difference in preference of each user.

  In the above-mentioned Non-Patent Document 1, it is described in the “Musubi” column on the last page that it is necessary to construct a system for adjusting the coefficient according to the preference of each user as a future problem. As described above, the conventional technology cannot estimate the subjective quality of the image according to the difference in the preference of each user.

  It is each user who determines whether the image quality is good or bad, and the actual user who uses the image communication service is not necessarily an average user. Therefore, in the estimation of “average subjective value (MOS)” Subjective quality is not estimated enough.

  For example, in the case of use of a TV phone or the like, a person image is mostly transmitted and received, and a subjective value difference due to a user difference is larger than a subjective value difference due to a difference in image content. That is, it is necessary to estimate the subjective value in consideration of user differences.

  However, according to the prior art, it is impossible to estimate the subjective quality of the image according to the difference in preference of each user.

  The present invention has been made in view of such circumstances, and when estimating the subjective quality of a user for an image, a new image that enables estimation of the subjective quality according to the difference in preference of each user The purpose is to provide subjective quality estimation technology.

To this end, the onset Ming, to realize that to estimate the subjective quality of the user of the image, also use it to determine the encoding / decoding parameters, (1) a plurality Generating means for generating a decoded image by encoding and decoding the original image to be processed by changing the encoding and / or decoding parameters, and (2) the decoded image generated by the generating means Execution means for inputting the subjective quality value of the subject to the decoded image and holding it in the storage means by inputting the subjective quality value shown to the subject and (3) decoding Selection means for selecting a plurality of reference images used for estimating the subjective quality of the user from among the decoded images based on the subjective quality value (read from the storage means) for the image; Selected reference image By subjective quality value (from the storage means) for cluster analysis for the classifying means for classifying the subject into a plurality of groups, (5) the subjective quality value of the reference image and the target variable, the subject Subjective quality values are obtained by performing linear or nonlinear multiple regression analysis using the encoding and / or decoding parameters used when generating the presented decoded images and the identifiers of the groups classified by the classification means as explanatory variables. Derivation means for deriving a relational expression (indicating the subjective quality evaluation characteristics of the user) between the parameter and the identifier of the group and the group, and (6) which of the groups classified by the classification means a user other than the subject belongs to comprising a determining means for determining further (7) the identifier of the determined group of the determination means, and the parameter used in the system, the derived relationship deriving means Based on the bets, calculating means for calculating the subjective quality value indicated by the user it is determined to belong to the group, or (8) the identifier of the determined group of the determination means, subjective requested by the user is determined to belong to the group and quality values, based on the derived relationship equation derivation means, configured with a decision means to determine the parameters for implementing the subjective quality value required of the user.

  When adopting this configuration, there may be a configuration in which the selection unit is not provided. In this case, the classification unit is configured to classify subjects into a plurality of groups based on the subjective quality value for the decoded image. The derivation means performs processing so as to derive a relational expression between the subjective quality value for the decoded image and the parameter and group identifier.

  Each of the above processing means can also be realized by a computer program. This computer program is provided by being recorded on an appropriate computer-readable recording medium or provided via a network, and is used when implementing the present invention. The present invention is realized by being installed and operating on a control means such as a CPU.

In thus configured present onset bright, for processing a plurality of original images, the original image of the encoding parameters and by changing the parameters of the decoding processing target to generate a decoded image by encoding decoding Then, by presenting the decoded images to the subject and inputting the subjective quality value indicated thereto, the subject's subjective quality value for the decoded images is input.

  For example, when using a hierarchical coding method such as MPEG4 FGS, the image quality can be changed greatly. Therefore, the base quantization parameter value is changed, the original image to be processed is encoded, and the number of planes is changed. Generate decoded images by decoding, present the decoded images to the subject, and input the subjective quality values shown for them, and input the subject's subjective quality values for those decoded images To do.

  Subsequently, based on the subjective quality value for the input decoded image, a plurality of reference images used for estimating the subjective quality of the user are selected from the decoded images.

  At this time, the degree of variation of the subjective quality value (standard deviation, etc.) is obtained for each decoded image, and based on this, the subject whose subjective quality evaluation is broken is selected from the decoded images. You may select an image.

  Further, the reference image may be selected by selecting the same number of decoded images from among the decoded images for each original image. Further, in this selection, the magnitude of the variation in subjective quality value for each decoded image may be selected. A reference image may be selected by obtaining (standard deviation or the like) and selecting a subject whose subjective quality evaluation is broken.

  Subsequently, the subjects are classified into a plurality of groups based on the subjective quality value for the selected reference image.

  For example, the subject is classified into a plurality of groups by performing a cluster analysis on the subjective quality value for the selected reference image.

  Subsequently, based on the subjective quality value for the reference image, the encoding and / or decoding parameters used to generate the reference image, and the classified group identifier, the subjective quality value, the parameter, and the group identifier Is derived.

  For example, a linear or nonlinear multiple regression analysis with the subjective quality value as the objective variable and the parameter and group identifier as the explanatory variable is performed to derive a relational expression between the subjective quality value and the parameter and group identifier. It is.

  At this time, as the group identifier, N groups represented by (N−1) dummy variables expressed by 1 or 0 may be used.

In this manner, in this onset bright, by changing the parameters of the encoding or decoding a plurality of original images and generates a decoded image by encoding decoding, subjective quality of the subject with respect to those of the decoded image Once the values are obtained, a plurality of reference images are selected from the decoded images based on the subjective quality values, and the subjects are grouped into a plurality of groups based on the subjective quality values for the reference images. In other words, a relational expression between the subjective quality value and the parameter and group identifier is derived.

  After deriving the relational expression in this way, when there is a request for estimation of the subjective quality value indicated by a user other than the subject, it is determined to which group the user belongs.

  For example, the list of subjective quality levels indicated by each group is presented to the user, and the group indicated by the subjective quality level selected from the list is identified to determine which group the user belongs to. In order to avoid this problem, the subjective quality value for the reference image is obtained from the user, and the subjective quality value indicated by the user is determined for each group. The distance from the center of gravity of the subjective quality value with respect to the reference image indicated by the subject belonging to the group to be obtained is obtained, and based on the distance, the group to which the user belongs is determined.

  Subsequently, the subjective quality value indicated by the user is calculated based on the identifier of the group to which the user belongs, the encoding and decoding parameters used in the system, and the relational expression obtained in the previous processing.

  In addition, after deriving the relational expression in this way, if there is a request for determining a parameter that realizes the subjective quality value requested by a user other than the subject, it is determined to which group the user belongs.

  For example, the list of subjective quality levels indicated by each group is presented to the user, and the group indicated by the subjective quality level selected from the list is identified to determine which group the user belongs to. In order to avoid this problem, the subjective quality value for the reference image is obtained from the user, and the subjective quality value indicated by the user is determined for each group. The distance from the center of gravity of the subjective quality value with respect to the reference image indicated by the subject belonging to the group to be obtained is obtained, and based on the distance, the group to which the user belongs is determined.

  Subsequently, a parameter for realizing the subjective quality value requested by the user is determined based on the identifier of the group to which the user belongs, the subjective quality value requested by the user, and the relational expression obtained in the previous processing.

In this manner, in this onset bright, after on the basis of the subjective quality value of the reference image shown in the subject to derive the relationship between the identifier of the subjective quality values and parameters and group, other than the subject If there is a request to estimate the subjective quality value indicated by the user, it is determined which group the user belongs to, and the relationship between the group identifier and the encoding and decoding parameters used in the system By substituting, the subjective quality value indicated by the user is calculated.

  Then, when there is a request for determining a parameter that realizes the subjective quality value requested by the user other than the subject, the group to which the user belongs is determined, and the identifier of the group and the subjective quality value requested by the user are determined. By substituting into the relational expression, a parameter for realizing the subjective quality value requested by the user is determined.

  For example, when a hierarchical encoding method such as MPEG4 FGS is used as the image encoding method, the base quantization parameter value is determined from the lowest subjective quality of the image communication service according to the relational expression, or the group to which the user belongs By determining the number of FGS planes to be decoded based on the attribute, the encoding parameters and decoding parameters that realize the subjective quality requested by the user are determined.

In accordance with this arrangement, according to this onset bright, when estimating the subjective quality of the user of the image, along with it it is possible to estimate the subjective quality according to a difference in taste of each user, the user It becomes possible to determine an encoding parameter and a decoding parameter that realize the required subjective quality.

  Here, the reference image is selected from a large number of decoded images because when a user other than the subject belongs to which group, a small number of reference images are selected for the user. This is because the subjective quality value may be input by presenting it. In addition, the selection of the reference image is performed so that the subject whose subjective quality evaluation is broken is selected from the decoded images so that the user's preferences can be classified with high accuracy.

  According to the present invention, subjective evaluation of an image used in an image communication service enables estimation of subjective quality of an image taking into account differences among users.

  At this time, the subjective image of the image communication service is subject to subjective evaluation, thereby reducing the burden on the user and estimating the subjective quality of the image considering the difference of each user Will be able to.

  And according to this invention, the encoding parameter and decoding parameter of the image which implement | achieve the subjective quality which a user requests | requires can be determined.

  Hereinafter, the present invention will be described in detail according to embodiments.

  In the following, an embodiment of the present invention will be described with reference to data and drawings, taking an MPEG4 FGS encoder / decoder as a specific example. The MPEG4 FGS encoder / decoder is introduced in the following document. In the following, in order to clarify the difference from the MOS indicating the average of the subjective quality values of all users, the subjective quality value for the image for each user is expressed as a user subjective quality value.

Literature: W. Li, "Overview of Fine Granularity Scalability in MPEG-4 Video Standard," IEEE Trans. Circuits Syst. Video Technol., Vol.11, no.3, pp.301-317, Mar.2001.
FIG. 1 illustrates an example of a device configuration of a subjective quality estimation formula deriving device 1 of the present invention that performs a process of deriving a subjective quality estimation formula used for estimating a user's subjective quality for an image.

  As shown in this figure, the subjective quality estimation formula deriving device 1 of the present invention includes an input / output device 2 serving as an interface with an operator, an original image database 10, a decoded image generating unit 11, a parameter change Information storage unit 12, decoded image database 13, subjective quality evaluation experiment execution unit 14, evaluation experiment result database 15, subjective quality estimation formula derivation unit 16, reference image ID storage unit 17, and cluster centroid storage unit 18 A subjective quality estimation formula storage unit 19, a parameter determination unit 20, and a user subjective quality value calculation unit 21.

  Here, 3-i (i = 1 to n) shown in the figure is a terminal device operated by the subject, and 4-i (i = 1 to m) is a terminal device operated by a user other than the subject, and the network The subjective quality estimation formula deriving device 1 of the present invention is connected via the above.

  The original image database 10 stores a plurality of original images to be processed. In this embodiment, it is assumed that three original images A, B, and C are stored.

  The decoded image generation unit 11 changes the base quantization parameter value and the number of FGS planes defined by the MPEG4 FGS image encoding method, and encodes and decodes the original image stored in the original image database 10. A decoded image is generated and stored in the decoded image database 13.

  Here, the number of FGS planes is the number of planes to be decoded counted from the most significant plane when the DCT coefficients to be FGS layer encoding in each frame of the moving image are expressed in binary numbers. For example, an FGS plane number of 1 means that only the most significant plane of each frame of a moving image is decoded. In addition, when the number of FGS planes is set to 4, if there are only three decoding target planes in the corresponding decoding target frame, up to three planes become decoding targets.

  The parameter change information storage unit 12 stores the values of the base quantization parameter value and the number of FGS planes to be changed by the decoded image generation unit 11. The decoded image generation unit 11 changes the base quantization parameter value and the number of FGS planes according to the value stored in the parameter change information storage unit 12 to encode and decode the original image stored in the original image database 10. Thus, a decoded image is generated.

  The subjective quality evaluation experiment execution unit 14 presents the decoded images stored in the decoded image database 13 to the subject, and inputs the user subjective quality value indicated for the decoded images. The user subjective quality value is input and stored in the evaluation experiment result database 15.

  The subjective quality estimation formula deriving unit 16 selects a plurality of reference images from among the decoded images that are subject of the subjective quality evaluation experiment execution unit 14 based on the user subjective quality values stored in the evaluation experiment result database 15. Then, after storing the ID information in the reference image ID storage unit 17, the subjects are classified into a plurality of clusters based on the user subjective quality values for the reference images, and the user subjectivity is determined for each cluster. The centroid of the quality value is obtained, and the centroid information is stored in the cluster centroid storage unit 18, and between the user subjective quality value, the changed encoding / decoding parameter of the decoded image generation unit 11 and the identifier of the classified cluster. A subjective quality estimation formula (indicating the subjective quality evaluation characteristic of the user) defined as a relational expression is derived, and information on the subjective quality estimation formula is stored in the subjective quality estimation formula storage unit 19 That.

  When there is a parameter determination request for realizing a user subjective quality value requested by a user other than the subject, the parameter determination unit 20 obtains the user subjective quality value for the reference image from the user, and indicates the user Based on the user subjective quality value and the center of gravity information of the cluster, the cluster to which the user belongs is specified, and based on the identifier of the specified cluster, the user subjective quality value requested by the user, and the subjective quality estimation formula Thus, the encoding / decoding parameters for realizing the user subjective quality value requested by the user are determined.

  When there is a user subjective quality value estimation request indicated by a user other than the subject, the user subjective quality value calculation unit 21 obtains the user subjective quality value for the reference image from the user, and the user subjectivity indicated by the user Based on the quality value and the centroid information of the cluster, the cluster to which the user belongs is specified, and the identifier of the specified cluster, the encoding / decoding parameters used in the system, and the subjective quality estimation formula are used. Based on this, the user subjective quality value indicated by the user is calculated.

  FIG. 2 illustrates an example of a processing flow executed by the decoded image generation unit 11, FIG. 3 illustrates an example of a processing flow executed by the subjective quality evaluation experiment execution unit 14, and FIG. 4 illustrates a subjective quality estimation formula. An example of a processing flow executed by the derivation unit 16 is illustrated, and FIG. 5 illustrates an example of a processing flow executed by the parameter determination unit 20.

  Next, processing executed by the subjective quality estimation formula deriving device 1 of the present invention configured as described above will be described according to these processing flows. First, processing executed by the decoded image generation unit 11 will be described according to the processing flow of FIG.

  When the decoded image generation unit 11 is activated in response to the generation request of the decoded image, first, as shown in the processing flow of FIG. 2, first, in step S10, the original image stored in the original image database 10 is stored. One unprocessed original image is selected and read.

  Subsequently, in step S11, one unprocessed parameter is selected from the encoding / decoding parameter values (base quantization parameter value, number of FGS planes) defined according to the storage information of the parameter change information storage unit 12. Thus, the encoding / decoding parameter value to be used is set.

  Subsequently, in step S12, a decoded image is generated by encoding and decoding the selected original image according to the set encoding / decoding parameter value, and is stored in the decoded image database 13. In step S13, all the codes are encoded. It is determined whether or not the normalization / decoding parameter value is set.

  When it is determined that all the encoding / decoding parameter values are not set according to the determination process in step S13, the process returns to step S11, and on the other hand, it is determined that all the encoding / decoding parameter values are set. If so, the process proceeds to step S14 to determine whether or not all original images have been selected.

  Then, according to the determination process of step S14, when it is determined that all original images have not been selected, the process returns to step S10. On the other hand, when it is determined that all original images have been selected, the process ends. To do.

  In this way, according to the process of the decoded image generating unit 11, the decoded image database 13 includes a plurality of images generated by changing the base quantization parameter value and the number of FGS planes for the original image A as shown in FIG. A plurality of decoded images generated by changing the base quantization parameter value and the number of FGS planes for the original image B, and changing the base quantization parameter value and the number of FGS planes for the original image C. A plurality of generated decoded images are stored.

  Next, processing executed by the subjective quality evaluation experiment execution unit 14 will be described according to the processing flow of FIG.

  When the subjective quality evaluation experiment execution unit 14 is activated in response to the execution request of the subjective quality evaluation experiment, first, as shown in the processing flow of FIG. One subject is selected from subjects who have not conducted subjective quality assessment experiments.

  Subsequently, in step S21, one unprocessed decoded image is selected from the decoded images stored in the decoded image database 13 and read. Subsequently, in step S22, the selected decoded image is presented to the selected subject via the terminal device 3-i, and the user subjective quality value for the decoded image is acquired. For example, the user subjective quality value indicated by the value of the five-level evaluation is acquired.

  Subsequently, in step S23, the acquired user subjective quality value is stored in the evaluation experiment result database 15, and in the subsequent step S24, it is determined whether or not all decoded images have been selected.

  When it is determined that all the decoded images are not selected according to the determination process in step S24, the process returns to step S21. On the other hand, when it is determined that all the decoded images are selected, the process proceeds to step S25. It is determined whether or not all subjects have been selected.

  Then, when it is determined that all the subjects have not been selected according to the determination processing of step S25, the processing returns to step S20, and when it is determined that all the subjects have been selected, the processing is terminated.

  In this way, according to the process of the subjective quality evaluation experiment execution unit 14, the evaluation experiment result database 15 indicates what user subjective quality value each subject has for each decoded image as shown in FIG. Is stored.

  Next, processing executed by the subjective quality estimation formula deriving unit 16 will be described according to the processing flow of FIG.

  When the subjective quality estimation formula deriving unit 16 is activated in response to the subjective quality estimation formula deriving request, first, as shown in the processing flow of FIG. The user subjective quality value of each subject with respect to the selected decoded image is read from the evaluation experiment result database 15.

  Subsequently, in step S31, the standard deviation of the read user subjective quality value is calculated. In subsequent step S32, it is determined whether or not all the decoded images are selected, and all the decoded images are not selected. Is determined, the process returns to step S30.

  In this way, by repeating the processing of step S30 to step S32, as shown in FIG. 8, the standard deviation is calculated for each decoded image.

  Subsequently, in step S33, for each original image, a reference image is set by selecting a predetermined number of decoded images according to the order of the standard deviation, and the ID information of the set reference image is used as the reference image. Store in the ID storage unit 17.

  For example, when one decoded image is selected for each original image, according to the standard deviation shown in FIG. 8, the decoded image for the original image A named image1, the decoded image for the original image B named image2, and the image3 The decoded image of the original image C is set as a reference image, and the ID information is stored in the reference image ID storage unit 17.

  Here, the reason why a decoded image having a large standard deviation is selected as a reference image is that the user's subjective quality evaluation is divided with respect to those decoded images. This is because the user's subjective quality evaluation characteristic can be derived in a clear form. In addition, the reason why the reference image is selected from the decoded images for each original image is that the user's subjective quality evaluation characteristics are different for each original image.

  Subsequently, in step S34, the user subjective quality value for the reference image of each subject is read from the evaluation experiment result database 15.

  For example, when the decoded images of image1, image2, and image3 are set as reference images, user subjective quality values for these reference images are read from the evaluation experiment result database 15 as shown in FIG. is there.

  Subsequently, in step S35, the subject is classified into a predetermined number of clusters by performing cluster analysis using the read user subjective quality value.

  Using the subject's user subjective quality value, the Euclidean distance between the subject and the subject can be defined. From now on, when there are a plurality of individuals, it is possible to use a cluster analysis method for classifying those individuals into clusters based on the Euclidean distance between the individuals. In this step S35, the read user subjective quality value is used. By performing cluster analysis, subjects are classified into a predetermined number of clusters.

  According to the processing of step S35, the subject can be classified into, for example, three clusters as shown in FIG.

  Subsequently, in step S36, (N-1) dummy variables expressed by 1 or 0 are assigned to N clusters, and the numbers of the classified clusters are converted into the dummy variables.

  For example, as shown in FIG. 11, a dummy variable (C1 = 0, C2 = 0) is assigned to cluster number 1 and (C1 = 0, C2 = 1) is assigned to cluster number 2. A dummy variable is assigned, a dummy variable (C1 = 1, C2 = 0) is assigned to the cluster number 3, and the number of each classified cluster is converted to this dummy variable.

  Subsequently, in step S37, the center of gravity of the subjective quality value with respect to the reference image is calculated for each cluster, and the calculated center of gravity information is stored in the cluster center of gravity storage unit 18.

  For example, as shown in FIG. 12, for the cluster with cluster number 1, an average value of user subjective quality values for the reference image image1 and an average value of user subjective quality values for the reference image image2 The center of gravity (2.666667, 2,2.666667) is calculated by calculating the average value of the user subjective quality values for the reference image image3, and the cluster of cluster number 2 is compared to the reference image image1. By calculating the average value of the user subjective quality values, the average value of the user subjective quality values for the reference image image2, and the average value of the user subjective quality values for the reference image image3 (3.7). , 2.8, 3.5), and for the cluster with cluster number 3, the average value of the user subjective quality value for the reference image image1 and the user subjective quality value for the reference image image2 By calculating the average value and the average value of the user subjective quality values with respect to the reference image image3, a centroid of (4.5, 4, 4.375) is calculated, and the calculated centroid information is used as the cluster centroid storage unit 18. It is stored in.

  Here, as shown in FIG. 12, from the size of the center of gravity, subjects belonging to the cluster with cluster number 1 are strict with respect to image quality, or subjects belonging to the cluster with cluster number 2 are normal with respect to image quality. In other words, subjects belonging to the cluster of cluster number 3 can specify a level for image quality that is unsatisfactory for image quality.

  Subsequently, in step S38, the dummy variable obtained in step S36 and the encoding / decoding parameters (base quantization parameter value, number of FGS planes) at the time of creation of the decoded image are used as explanatory variables, and the user subjectivity with respect to the reference image is determined. A subjective quality estimation formula is derived by performing linear or non-linear multiple regression analysis using the quality value as an objective variable, the subjective quality estimation formula information is stored in the subjective quality estimation formula storage unit 19, and the processing ends. To do.

According to the processing of step S38, if GP is a user subjective quality value, QP is a base quantization parameter value, FP is the number of FGS planes to be decoded, and C1 and C2 are dummy variables, for example,
GP = 3.5−0.1 × QP + 0.5 × FP + 0.5 × C1 + 0.2 × C2
That is, the subjective quality estimation formula is derived.

The subjective quality estimation formula deriving unit 16 may store the subjective quality estimation formula derived in this way in the subjective quality estimation formula storage unit 19 as it is, but by defining the minimum image quality BGP,
GP = BGP + FGP Formula (1)
BGP = 3.5−0.1 × QP (2)
FGP = 0.5 × FP + 0.5 × C1 + 0.2 × C2 Formula (3)
It may be modified as described above and stored in the subjective quality estimation formula storage unit 19.

  In the case of performing such cluster classification, in the prior art, an expression such as a subjective quality estimation expression is derived for each cluster. For example, when there are three clusters, expressions such as three subjective quality estimation expressions are derived.

  However, when such a method is used, the user cannot be handled uniformly when estimating the subjective quality evaluation of the user.

  Therefore, the subjective quality estimation formula deriving unit 16 derives a subjective quality estimation formula that can handle users belonging to each cluster in a unified manner by multiple regression analysis by making the clusters dummy variables.

  In accordance with this configuration, according to the present invention, for example, when there are three clusters, three subjective quality estimation formulas are required according to the conventional cluster classification technique. It will be possible.

  Further, when performing cluster classification, it is generally a problem how to use classification indices. Cluster classification using user attributes such as gender and age does not necessarily reflect the evaluation tendency of the image quality of each user.

  Therefore, the subjective quality estimation formula deriving unit 16 selects a decoded image having a large standard deviation as a reference image in order to derive a subjective quality estimation formula reflecting the image quality evaluation tendency of the user, thereby further improving the image quality evaluation. Cluster classification is performed using an affected reference image.

  According to this configuration, according to the present invention, it is possible to derive a subjective quality estimation formula corresponding to the user's evaluation tendency.

  Next, processing executed by the parameter determination unit 20 will be described according to the processing flow of FIG.

  When there is a parameter determination request for realizing the user subjective quality value requested by the user via the terminal 4-i operated by a user other than the subject, the parameter determination unit 20 as shown in the processing flow of FIG. First, in step S40, the reference image ID storage unit 17 is referred to obtain which is the reference image, the reference image is read from the decoded image database 13, and the reference image is stored in the terminal device. The user subjective quality value for the reference image is obtained by presenting to the user via 4-i. For example, a user subjective quality value indicated by, for example, a five-level evaluation value acquired by the subjective quality evaluation experiment execution unit 14 is acquired.

  Subsequently, in step S41, the Euclidean distance between the acquired user subjective quality value and the centroid of each cluster stored in the cluster centroid storage unit 18 is calculated, and the cluster having the minimum Euclidean distance is specified. The cluster to which the user belongs is specified.

  Subsequently, in step S42, the identified cluster number is converted into a dummy variable. As described above, when the cluster number is 1, it is converted into a dummy variable (C1 = 0, C2 = 0), and when the cluster number is 2, (C1 = 0, C2 = 1). If the cluster number is 3, it is converted to a dummy variable (C1 = 1, C2 = 0).

  Subsequently, in step S43, the user subjective quality value requested by the user is acquired. The user subjective quality value requested by the user is acquired via the terminal 4-i.

  Subsequently, in step S44, the subjective quality estimation formula is obtained from the subjective quality estimation formula storage unit 19, and the converted dummy variable and the acquired user subjective quality value requested by the user are substituted for the subjective quality estimation formula. As a result, the encoding / decoding parameter that realizes the user subjective quality value requested by the user is determined, and the process ends.

For example,
GP = BGP + FGP Formula (1)
BGP = 3.5−0.1 × QP (2)
FGP = 0.5 × FP + 0.5 × C1 + 0.2 × C2 Formula (3)
If the minimum image quality is set to BGP = 3 when the subjective quality estimation formula is determined, QP = 5 is obtained according to formula (2). In this case, if a user belonging to a cluster that is “strict” for image quality evaluation (C1 = 0, C2 = 0) requests GP = 4, FP = 2 is determined according to equation (1). That is, 2 is determined as the number of FGS planes in order to realize the user subjective quality value requested by the user belonging to the “strict” cluster for the image quality evaluation.

  When a user belonging to a “sweet” cluster (C1 = 1, C2 = 0) for image quality evaluation requests GP = 4, FP = 1 is determined according to equation (1). That is, 1 is determined as the number of FGS planes in order to realize the user subjective quality value requested by the user belonging to the “sweet” cluster for the image quality evaluation.

  As can be seen from the above, according to the subjective quality estimation formula deriving device 1 of the present invention, the base quantization parameter QP can be determined only by specifying the minimum image quality that can be provided and the user subjective quality value required by each user. And the number of FGS planes FP in consideration of the difference between users can be determined.

  FIG. 13 shows a processing flow of processing executed by the subjective quality estimation formula deriving device 1 of the present invention described above.

  That is, as shown in the processing flow of FIG. 13, the subjective quality estimation formula deriving device 1 of the present invention first encodes and decodes an original image while changing encoding / decoding parameters in step S50. Then, a decoded image used for the subjective quality evaluation experiment is generated.

  Subsequently, in step S51, a subjective quality evaluation experiment is performed on the subject using the generated decoded image, and a user subjective quality value for the decoded image is acquired.

  Subsequently, in step S52, the standard deviation of the user subjective quality value is calculated for each decoded image, and based on the standard deviation, a predetermined number of decoded images are selected according to the order of the standard deviation for each original image. Set the reference image.

  Subsequently, in step S53, the subject is classified into a plurality of clusters by performing cluster analysis based on the user subjective quality value with respect to the reference image indicated by each subject.

  Subsequently, in step S54, the center of gravity of the user subjective quality value with respect to the reference image is calculated for each cluster.

  Subsequently, in step S55, the numbers of the classified clusters are converted into dummy variables expressed by 1 and 0.

  Subsequently, in step S56, a subjective quality estimation formula is derived by performing multiple regression analysis using dummy variables and encoding / decoding parameters as explanatory variables and user subjective quality values for reference images as objective variables. .

  Subsequently, in step S57, an encoding / decoding parameter for realizing a user subjective quality value requested by the user is entered, and a subjective quality evaluation experiment is performed using a reference image for a user other than the subject. The user subjective quality value for the reference image is acquired.

  Subsequently, in step S58, the cluster to which the user belongs is specified based on the Euclidean distance between the user subjective quality value with respect to the reference image indicated by the user and the cluster centroid, and the number of the cluster is set as a dummy variable. Convert to

  Subsequently, in step S59, the user subjective quality value requested by the user is acquired via the terminal device 4-i.

  Finally, in step S60, the user subjective quality value requested by the user is realized based on the derived subjective quality estimation formula, the dummy variable for the cluster number to which the user belongs, and the user subjective quality value requested by the user. The encoding / decoding parameters to be determined are determined, and the process is terminated.

  In this way, according to the subjective quality estimation formula deriving device 1 of the present invention, a user other than the subject can input the user subjective quality value with respect to the preset reference image, and the user's It becomes possible to determine an encoding parameter and a decoding parameter of an image that realizes the required subjective quality.

  In the processing flow of FIG. 13, it has been described that the parameter determination unit 20 operates in steps S57 to S60. However, the user subjective quality value calculation unit 21 may operate instead of the parameter determination unit 20.

  Next, processing executed by the user subjective quality value calculation unit 21 will be described according to the processing flow of FIG.

  When there is a user subjective quality value estimation request indicated by the user via the terminal 4-i operated by a user other than the subject, the user subjective quality value calculation unit 21 first starts as shown in the processing flow of FIG. In step S70, the reference image ID storage unit 17 is referred to obtain which one of the reference images, the reference images are read from the decoded image database 13, and the reference images are read from the terminal device 4- The user subjective quality value for the reference image is obtained by presenting to the user via i. For example, a user subjective quality value indicated by, for example, a five-level evaluation value acquired by the subjective quality evaluation experiment execution unit 14 is acquired.

  Subsequently, in step S71, the Euclidean distance between the acquired user subjective quality value and the centroid of each cluster stored in the cluster centroid storage unit 18 is calculated, and the cluster having the minimum Euclidean distance is specified. The cluster to which the user belongs is specified.

  In step S72, the identified cluster number is converted into a dummy variable. As described above, when the cluster number is 1, it is converted into a dummy variable (C1 = 0, C2 = 0), and when the cluster number is 2, (C1 = 0, C2 = 1). If the cluster number is 3, it is converted to a dummy variable (C1 = 1, C2 = 0).

  Subsequently, in step S73, the encoding / decoding parameters currently used by the image communication service are acquired via the input / output device 2.

  Subsequently, in step S74, a subjective quality estimation formula is obtained from the subjective quality estimation formula storage unit 19, and the converted dummy variable and the obtained encoding / decoding parameters are substituted into the subjective quality estimation formula. The user subjective quality value indicated by the user is calculated, and the process ends.

For example,
GP = 3.5−0.1 × QP + 0.5 × FP + 0.5 × C1 + 0.2 × C2
Is obtained by substituting the converted dummy variables C1 and C2 and the obtained encoding / decoding parameters QP and FP into the subjective quality estimation formula. The user subjective quality value GP indicated by the user is calculated.

  In this way, according to the subjective quality estimation formula deriving device 1 of the present invention, a user other than the subject can input the user subjective quality value with respect to the preset reference image, and the user's The user subjective quality value shown can be calculated.

  Although the present invention has been described according to the illustrated embodiment, the present invention is not limited to this. For example, in the embodiment, the burden on the user is obtained by selecting a reference image from the generated decoded images and using the selected reference image as an image for obtaining a subjective quality value from a user other than the subject. However, it is also possible to adopt a configuration in which the reference image is not selected.

  In the embodiment, the subjective quality value for the reference image is obtained from a user other than the subject, and the cluster to which the user belongs is automatically specified based on the subjective quality value. A configuration in which information (characteristic information and centroid information as shown in FIG. 12) indicating whether or not a cluster is classified is presented, and a cluster to which the user belongs is selected from the information. It is also possible to take.

It is a device block diagram of the subjective quality estimation formula deriving device of the present invention. It is a processing flow which a decoded image generation part performs. It is a processing flow which a subjective quality evaluation experiment execution part performs. It is the processing flow which a subjective quality estimation formula derivation part performs. It is a processing flow which a parameter determination part performs. It is explanatory drawing of the decoded image stored in a decoded image database. It is explanatory drawing of the experimental result data stored in an evaluation experimental result database. It is explanatory drawing of the standard deviation which a subjective quality estimation formula derivation | leading-out part calculates. It is a figure which shows an example of the user subjective quality value with respect to a reference | standard image. It is a figure which shows an example of the classification result by cluster analysis. It is explanatory drawing of a dummy variable. It is explanatory drawing of the gravity center information of a cluster. It is a processing flow which the subjective quality estimation type | formula derivation | leading-out apparatus of this invention performs. It is a processing flow which a user subjective quality value calculation part performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Subjective quality estimation formula derivation device 2 Input / output device 3 Terminal device 4 Terminal device 10 Original image database 11 Decoded image generation unit 12 Parameter change information storage unit 13 Decoded image database 14 Subjective quality evaluation experiment execution unit 15 Evaluation experiment result database 16 Subjective Quality estimation formula derivation unit 17 Reference image ID storage unit 18 Cluster centroid storage unit 19 Subjective quality estimation formula storage unit 20 Parameter determination unit 21 User subjective quality value calculation unit

Claims (9)

A subjective quality estimation formula derivation method executed by a subjective quality estimation formula derivation device that derives and uses a formula for estimating the subjective quality of a user for an image,
A process of generating a decoded image by encoding and decoding an original image to be processed with a plurality of original images as processing targets and changing encoding and / or decoding parameters;
Presenting the decoded image to the subject and inputting the subjective quality value indicated thereto, and inputting the subject's subjective quality value for the decoded image and holding it in the storage means;
Classifying the subjects into a plurality of groups by clustering the subjective quality values; and
By performing linear or non-linear multiple regression analysis with the subjective quality value as an objective variable and the parameters used for generating the decoded image presented to the subject and the identifier of the group as explanatory variables, the subjective quality value and Deriving a relation between the parameter and the identifier of the group ;
The decoded image is presented to a user other than the subject, and the subjective quality value indicated for the decoded image is input, and the user other than the subject is in the group according to the distance from the center of gravity of the subjective quality value in the subject group. The group to which the subjective quality level selected from the list is presented, and a list of subjective quality levels indicated by each group is presented to a user other than the subject. A process of determining which of the groups a user other than the subject belongs,
A step of calculating a subjective quality value indicated by the user using the relational expression from the determined group identifier and the parameter used in the system,
A method for deriving a characteristic subjective quality estimation formula . A subjective quality estimation formula derivation method executed by a subjective quality estimation formula derivation device that derives and uses a formula for estimating the subjective quality of a user for an image ,
A process of generating a decoded image by encoding and decoding an original image to be processed with a plurality of original images as processing targets and changing encoding and / or decoding parameters;
Presenting the decoded image to the subject and inputting the subjective quality value indicated thereto, and inputting the subject's subjective quality value for the decoded image and holding it in the storage means;
Classifying the subjects into a plurality of groups by clustering the subjective quality values; and
By performing linear or non-linear multiple regression analysis with the subjective quality value as an objective variable and the parameters used for generating the decoded image presented to the subject and the identifier of the group as explanatory variables, the subjective quality value and Deriving a relation between the parameter and the identifier of the group;
The decoded image is presented to a user other than the subject, and the subjective quality value indicated for the decoded image is input, and the user other than the subject is in the group according to the distance from the center of gravity of the subjective quality value in the subject group. The group to which the subjective quality level selected from the list is presented, and a list of subjective quality levels indicated by each group is presented to a user other than the subject. A process of determining which of the groups a user other than the subject belongs,
Acquiring a subjective quality value required by a user other than the subject; and
A step of determining encoding and / or decoding parameters for realizing a subjective quality value requested by a user other than the subject using the relational expression from the determined group identifier and the acquired subjective quality value; Having
A method for deriving a characteristic subjective quality estimation formula. In the subjective quality estimation formula deriving method according to claim 1 or 2,
In the process of classifying the subjects into a plurality of groups, the degree of variation in the subjective quality value is obtained for each decoded image, and a predetermined number of decoded images are used as a reference image in descending order of variation from the decoded images. Selecting and classifying the subjects into a plurality of groups by clustering the subjective quality values for the selected reference image;
In the process of determining which of the groups a user other than the subject belongs, a decoded image presented to a user other than the subject is used as the reference image .
A method for deriving a characteristic subjective quality estimation formula . In the subjective quality estimation formula derivation method according to any one of claims 1 to 3 ,
In the process of deriving the relational expression, as the identifier of the group, N groups represented by (N−1) dummy variables expressed by 1 or 0 can be used.
A method for deriving a characteristic subjective quality estimation formula . In the subjective quality estimation formula deriving method according to any one of claims 1 to 4 ,
When using a hierarchical coding method as the coding parameter, using a quantization parameter value of a base layer ,
A method for deriving a characteristic subjective quality estimation formula . In the subjective quality estimation formula deriving method according to any one of claims 1 to 5 ,
As the decoding parameter, when using a hierarchical coding scheme, using the number of planes of the enhancement layer ,
A method for deriving a characteristic subjective quality estimation formula . A subjective quality estimation formula derivation device that derives and uses a formula for estimating the subjective quality of a user for an image,
Means for generating a decoded image by encoding and decoding the original image to be processed with a plurality of original images as processing targets, changing the encoding and / or decoding parameters;
Means for presenting the decoded image to the subject and inputting the subjective quality value indicated for the subject to input the subject's subjective quality value for the decoded image and holding it in the storage means;
Means for classifying the subjects into a plurality of groups by clustering the subjective quality values;
By performing linear or non-linear multiple regression analysis with the subjective quality value as an objective variable and the parameters used for generating the decoded image presented to the subject and the identifier of the group as explanatory variables, the subjective quality value and Means for deriving a relational expression between the parameter and the identifier of the group ;
The decoded image is presented to a user other than the subject, and the subjective quality value indicated for the decoded image is input, and the user other than the subject is in the group according to the distance from the center of gravity of the subjective quality value in the subject group. The group to which the subjective quality level selected from the list is presented, and a list of subjective quality levels indicated by each group is presented to a user other than the subject. Means for determining which of the groups a user other than the subject belongs,
Means for calculating a subjective quality value indicated by the user using the relational expression from the determined group identifier and the parameter used in the system,
Subjective quality estimation formula derivation device . A subjective quality estimation formula derivation device that derives and uses a formula for estimating the subjective quality of a user for an image ,
Means for generating a decoded image by encoding and decoding the original image to be processed with a plurality of original images as processing targets, changing the encoding and / or decoding parameters;
Means for presenting the decoded image to the subject and inputting the subjective quality value indicated for the subject to input the subject's subjective quality value for the decoded image and holding it in the storage means;
Means for classifying the subjects into a plurality of groups by clustering the subjective quality values;
By performing linear or non-linear multiple regression analysis with the subjective quality value as an objective variable and the parameters used for generating the decoded image presented to the subject and the identifier of the group as explanatory variables, the subjective quality value and Means for deriving a relational expression between the parameter and the identifier of the group;
The decoded image is presented to a user other than the subject, and the subjective quality value indicated for the decoded image is input, and the user other than the subject is in the group according to the distance from the center of gravity of the subjective quality value in the subject group. The group to which the subjective quality level selected from the list is presented, and a list of subjective quality levels indicated by each group is presented to a user other than the subject. Means for determining which of the groups a user other than the subject belongs,
Means for obtaining a subjective quality value requested by a user other than the subject;
Means for determining encoding and / or decoding parameters for realizing a subjective quality value requested by a user other than the subject using the relational expression from the determined group identifier and the acquired subjective quality value; Having
Subjective quality estimation formula derivation device. It claims 1 to subjective quality estimation equation derivation processing program for executing the subjective quality estimation equation derivation method according to computer in any one of 6.

Images