JP7783482B2 - Model generation device and model generation method - Google Patents

Description

One aspect of the present invention relates to a model generation device that generates a learning model for performing drawing searches.

For example, in the field of plant engineering, it is necessary to handle large amounts of drawings (e.g., past drawings). For this reason, technology has been proposed to efficiently search for desired drawings from among a large number of drawings.

As an example, Patent Document 1 discloses technology aimed at improving user convenience when searching for drawings. Specifically, the technology in Patent Document 1 uses machine learning to generate a learning model for performing drawing searches.

Japanese Patent Application Laid-Open No. 2021-12413

One aspect of the present invention aims to improve the quality of learning models for drawing searches compared to conventional methods.

To solve the above problem, one aspect of the present invention provides a model generation device that generates a learning model for searching for at least one drawing corresponding to a target drawing from among a plurality of search target drawings. The model generation device includes: an acquisition unit that analyzes the plurality of search target drawings to acquire a content parameter set including a plurality of content parameters related to the content described in each of the plurality of search target drawings; (i) a determination unit that calculates a multicollinearity evaluation value between two different content parameters from among the plurality of content parameters included in the content parameter set for each combination pattern of the two different content parameters, and (ii) determines a content parameter to be deleted from among the plurality of content parameters based on the multicollinearity evaluation value; and a learning unit that generates the learning model based on a pruned content parameter set obtained by deleting the content parameter to be deleted from the content parameter set.

A model generation method according to one aspect of the present invention is a model generation method for generating a learning model for searching for at least one drawing corresponding to a target drawing from among a plurality of search target drawings, and includes: an acquisition step of acquiring a content parameter set including a plurality of content parameters related to the content described in each of the plurality of search target drawings by analyzing the plurality of search target drawings; (i) a determination step of calculating a multicollinearity assessment value between two different content parameters from among the plurality of content parameters included in the content parameter set for each combination pattern of the two different content parameters, and (ii) a determination step of determining, from among the plurality of content parameters, content parameters to be deleted based on the multicollinearity assessment value; and a learning step of generating the learning model based on a pruned content parameter set obtained by deleting the content parameters to be deleted from the content parameter set.

One aspect of the present invention makes it possible to improve the quality of learning models for drawing searches compared to conventional methods.

FIG. 1 is a block diagram showing a configuration of a main part of an information processing system according to a reference embodiment. FIG. 10 is a diagram showing an example of a past property/drawing correspondence table in a reference form. FIG. 10 is a diagram showing an example of a past drawing content parameter table in the reference embodiment. FIG. 10 is a diagram showing an example of a content parameter setting table in an initial state in the reference embodiment. FIG. 10 is a diagram showing an example of the number of content parameters after preprocessing in each data set in the reference embodiment. FIG. 10 is a diagram illustrating a schematic example of the data structure of dataset 1 in the reference embodiment. 10A and 10B are diagrams showing examples of a normalization data table and a standardization data table in the reference embodiment. FIG. 10 is a diagram illustrating an example of a dataset/preprocessing method correspondence table in the reference embodiment. FIG. 10 is a diagram illustrating some of a plurality of data sets generated by a learning preprocessing unit in the reference embodiment. FIG. 10 is a diagram showing an example of an evaluation result table in the reference embodiment. FIG. 10 is a diagram showing an example of a content parameter setting table after updating in the reference embodiment. FIG. 10 is a diagram showing an example of a new drawing content parameter table in the reference embodiment. FIG. 10 is a diagram showing an example of a post-preprocessing new drawing content parameter table in the reference embodiment. FIG. 10 is a diagram illustrating some of a plurality of raw value/label value conversion tables in a reference embodiment. 10 illustrates an example of a raw value/label value conversion integrated table in a reference embodiment. 1 is a block diagram showing a configuration of a main part of an information processing system according to a first embodiment. FIG. 10 is a diagram showing an example of a table of coefficients of determination and the number of high-risk coefficients of determination in the initial state in the first embodiment. FIG. 18 is a diagram showing an example of a coefficient of determination/high risk coefficient of determination number table obtained by updating the coefficient of determination/high risk coefficient of determination number table of FIG. 17. FIG. 19 is a diagram showing an example of a coefficient of determination/high risk coefficient of determination number table obtained by updating the coefficient of determination/high risk coefficient of determination number table of FIG. 18. FIG. 10 is a diagram showing an example of a final determination coefficient/high risk determination coefficient number table obtained as a result of repeated updates. FIG. 21 is a diagram showing an example of a pruned dataset 1 corresponding to the table of coefficients of determination and the number of high-risk coefficients of determination in FIG. 20. FIG. 10 is a block diagram showing the configuration of a main part of an information processing system according to a second embodiment.

[Reference form]
Prior to describing the information processing system 100 of the first embodiment, an information processing system 100s will be described as a reference embodiment. For convenience of explanation, components having the same functions as those described in the reference embodiment will be denoted by the same reference numerals in the following embodiments, and their descriptions will not be repeated. For simplicity, descriptions of matters similar to those in known technologies will also be omitted as appropriate.

Please note that the numerical values described below in this specification are merely examples. In this specification, the expression "A to B" for two numbers A and B means "greater than or equal to A and less than or equal to B" unless otherwise specified.

(Overview of information processing system 100s)
1 is a block diagram showing the configuration of a main part of an information processing system 100s. The information processing system 100s includes an information processing device 1s, a previous property plan DB (Database) 91, and a new property plan DB 92.

The information processing device 1s includes a control device 10s, an input unit 71, a display unit 72, and a memory unit 80. The control device 10s includes a learning device 11s and a drawing search device 12. In this specification, "property" refers to, for example, a "site" in plant engineering.

The information processing device 1s only needs to be communicatively connected to the past property plan DB 91 and the new property plan DB 92. Therefore, unlike the example in Figure 1, at least one of the past property plan DB 91 and the new property plan DB 92 may be provided inside the information processing device 1s.

Drawings to be searched for by the information processing device 1s (more specifically, the control device 10s) (search target drawings) include, for example, specification drawings, design drawings, and production drawings. Search target drawings may also include specifications, design documents, and estimates. In this way, search target drawings are not limited to documents containing the word "drawing." As an example, search target drawings include any type of document related to project plans in the plant engineering field.

However, as will be clear to those skilled in the art, the information processing device according to one aspect of the present invention can also be applied to searching for drawings in fields other than plant engineering. Drawings according to one aspect of the present invention may be drawings from which content parameters can be obtained using the information processing device.

The control device 10s controls all components of the information processing device 1s. The memory unit 80 stores various data and programs used in the processing of the control device 10s. As described below, the control device 10s uses machine learning to search for at least one drawing corresponding to the target drawing (e.g., drawing ND) from multiple search target drawings (e.g., past drawings a1 to MN).

The input unit 71 accepts user operations (user operations). The display unit 72 displays various data. As an example, the display unit 72 may display data indicating the search results of the control device 10s. The input unit 71 and display unit 72 may be provided as an integrated unit. For example, the input unit 71 and display unit 72 can be integrated by using a touch panel.

(Past property drawings DB91)
The past property drawing DB 91 stores drawings (strictly speaking, drawing data) for each past property (each existing property). In the following explanation, "drawing data of drawing A (a certain drawing)" will be abbreviated as simply "drawing A" as appropriate. Also, "drawing number of drawing A" will be abbreviated as simply "drawing A" as appropriate.

In the reference embodiment, the past property drawing DB91 stores drawings for each of a number M of different properties. M is an integer greater than or equal to 1. Hereinafter, the jth property will also be referred to as "property j." j is an integer greater than or equal to 1 and less than or equal to M.

In addition, the past property drawing DB91 stores N different drawings (N types) for each of properties 1 to M. N is an integer greater than or equal to 1. Hereinafter, the i-th drawing for property j will also be referred to as "drawing (i, j)." Furthermore, the i-th drawing (i-th type) for each property will also be collectively referred to as drawing i. i is an integer greater than or equal to 1 and less than or equal to N.

As described above, a total of T drawings are stored in the past property drawing DB91. In the reference example, T = M x N. T is an integer greater than or equal to 2. In other words, at least one of M and N is greater than or equal to 1.

Specifically, in the past property drawing DB91, similar to Patent Document 1, each type of drawing number is listed by property number in the form of the past property/drawing correspondence table TB1 shown in Figure 2. Hereinafter, the past property/drawing correspondence table TB1 will also be abbreviated as "TB1." Other elements will also be abbreviated in the same way as appropriate. The cell in the i-th row and j-th column of TB1 indicates the drawing number of drawing (i, j).

In the example of Figure 2, for convenience, properties 1 to 3 are also referred to as properties A to C, respectively. In the example of Figure 2, the first type of drawing (Drawing 1) is an outline drawing, the second type of drawing (Drawing 2) is an assembly drawing, and the third type of drawing (Drawing 3) is a foundation drawing. Furthermore, the Nth type of drawing (Drawing N) is a configuration drawing.

For simplicity's sake, drawings (1,1) to (N,1) in the example of Figure 2 (i.e., each of the N types of drawings for Property A) will also be referred to as drawings a1 to aN. For example, drawings a1 to aN in the example of Figure 2 refer to the outline drawing to the configuration drawing of Property A, respectively. The same notation will be used for each drawing for other properties.

As described above, the past property drawing DB91 stores a total of T drawings, from drawing a1 to drawing MN. Hereinafter, drawings a1 to MN will be collectively referred to as past drawings. Past drawings are an example of drawings to be searched. For this reason, drawings a1 to MN will also be referred to as a group of drawings to be searched.

In addition, in this specification, any one drawing from multiple past drawings (drawings to be searched) is also referred to as a candidate drawing. As an example, we will mainly illustrate the processes when drawing a1 (exterior drawing of property A) is the candidate drawing. Explanations of the processes for other drawings will be omitted where appropriate, but they are the same as those for drawing a1.

(New property drawing DB92)
The new property drawing DB 92 stores drawings (hereinafter collectively referred to as new drawings) relating to new properties (e.g., at least one property that is scheduled to be constructed) as a new property data set. In this specification, one new property (property T) included in the new property data set will be described.

As an example, the new property drawing DB92 stores N different drawings (outline drawings to configuration drawings) of the same type for property T as for previous properties. In this specification, the outline drawing for property T is used as an example of a new drawing. Hereinafter, the outline drawing for property T will be referred to as drawing ND. Drawing ND in the reference form is assumed to be the same as in Patent Document 1.

(Learning device 11s)
The learning device 11s includes a past drawing data acquisition unit 111, a past drawing content parameter acquisition unit 112 (candidate drawing content parameter acquisition unit, search target drawing content parameter acquisition unit, acquisition unit), a learning preprocessing unit 114 (preprocessing unit), and a learning model generation unit 113s. Based on the drawings a1 to MN, the learning device 11s generates a learning model for drawing search by the drawing search device 12s. For this reason, the learning device 11s may be referred to as a model generation device. An example of the processing flow of the learning device 11s is described below.

(Acquisition of candidate drawings)
In the past property drawing DB91, drawings a1 to MN are pre-sorted by drawing type according to TB1. Therefore, for example, the past drawing data acquisition unit 111 acquires each exterior drawing from the past property drawing DB91 in the order of "drawing a1 → b1 → ... → M1." Next, the past drawing data acquisition unit 111 acquires each assembly drawing from the past property drawing DB91 in the order of "drawing a2 → b2 → ... → M2." Finally, the past drawing data acquisition unit 111 acquires each configuration drawing from the past property drawing DB91 in the order of "drawing aN → bN → ... → MN." These drawings a1 to MN in the reference embodiment are assumed to be the same as those in Patent Document 1.

In the above example, the past drawing data acquisition unit 111 first references the cell in row 1, column 1 of TB1. Then, the past drawing data acquisition unit 111 acquires the drawing (1,1) corresponding to the above cell, i.e., drawing a1, from the past property drawing DB 91. The past drawing data acquisition unit 111 supplies the acquired drawing a1 to the past drawing content parameter acquisition unit 112.

(Obtaining content parameter sets corresponding to candidate drawings)
The previous drawing content parameter acquisition unit 112 acquires a content parameter set corresponding to the drawing a1 by analyzing the drawing a1 (more specifically, by performing a syntax analysis on the OCR-processed drawing a1, focusing on the k-th specific character string described below), in the same manner as in Patent Document 1. The content parameter set is a data set indicating the k-th content parameter (hereinafter, Ak) associated with the k-th specific character string. The first to L-th content parameters are also collectively referred to as content parameters.

In this specification, a specific character string (string) that is preset for each drawing is referred to as a specific character string. In the reference embodiment, it is assumed that L (L is an integer greater than or equal to 2) different kth specific character strings are preset. In the following, the kth specific character string is referred to as the kth specific character string. k is an integer greater than or equal to 1 and less than or equal to L. In the following explanation, an example is given in which the first specific character string is set to "voltage value," the second specific character string is set to "current value," the third specific character string is set to "OR," and the Lth specific character string is set to "open."

Content parameters are quantities associated with the content of a drawing (specifically, the content of a specific character string). Therefore, content parameters can be considered to be a type of data that quantifies (quantifies) the content of the drawing. For this reason, content parameters are used as indicators of the content of a drawing.

In order to distinguish them from the target drawing content parameters described below, the content parameters of the search target drawing (past drawing) are also referred to as search target drawing content parameters. Furthermore, the kth content parameter of the search target drawing is also referred to as the kth content parameter of the search target drawing. However, in the following explanation, unless otherwise specified, content parameters refer to past drawing content parameters. Similarly, unless otherwise specified, content parameter set refers to past drawing content parameter set.

In this specification, Ak of drawing (i, j) is also referred to as Ak(i, j). As described above, the past drawing content parameter acquisition unit 112 sets Ak(i, j) based on the analysis results for drawing (i, j) (more specifically, the detection results of specific character strings in drawing (i, j)).

As described above, the past drawing content parameter acquisition unit 112 sets A1 to AL for drawing a1. As described above, the past drawing content parameter acquisition unit 112 acquires the content parameters of the candidate drawing by analyzing the candidate drawing. For this reason, the past drawing content parameter acquisition unit 112 is also referred to as the candidate drawing content parameter acquisition unit.

The past drawing content parameter acquisition unit 112 also performs similar processing for other past drawings. That is, the past drawing content parameter acquisition unit 112 sets A1 to AL for each of drawings a1 to MN.

Then, the past drawing content parameter acquisition unit 112 generates a past drawing content parameter table TB2 that indicates A1 to AL of each of drawings a1 to MN, i.e., A1(1,1) to AL(M,N).

TB2 contains the i-th sub-table TB2-i in the past drawing content parameter table. Figure 3 shows TB2-1 as an example. TB2-i is a table showing A1 to AL for each of drawing i. TB2-1 shows A1 to AL for each of drawings a1 to M1 (exterior drawings of properties A to M). TB2 is made up of a set of N sub-tables, TB2-1 to TB2-N. In this way, in the reference embodiment, a sub-table is created for each drawing type. These sub-tables in the reference embodiment are assumed to be the same as those in Patent Document 1.

TB2 shows the number (an example of an identifier) of each of multiple past drawings (e.g., drawing a1) and the correspondence between A1 to AL of each of the multiple past drawings. The learning model generation unit 113s then acquires TB2 as training data. As an example, the learning model generation unit 113s may generate a learning model by performing multinomial logistic regression using the training data. However, as will be clear from the explanation below, the machine learning algorithm according to one aspect of the present invention is not limited to this example, and other well-known algorithms may be applied. Note that the numbers (identifiers) of each past drawing shown in TB2 are used as correct answer data in machine learning.

In the reference embodiment, the past drawing content parameter acquisition unit 112 acquires the kth content parameter and also acquires information indicating the variable type of the kth content parameter (kth content parameter variable type information). The kth content parameter variable type information is information indicating the variable type (data type) of the kth content parameter. Specifically, the kth content parameter variable type information is information indicating whether the kth content parameter is a qualitative variable (hereinafter referred to as VL) or a quantitative variable (hereinafter referred to as VN). Hereinafter, the first to kth content parameter variable type information will be collectively referred to as content parameter variable type information.

As an example, the past drawing content parameter acquisition unit 112 may acquire content parameter variable type information based on the results of the above-mentioned syntax analysis. For example, as described above, the past drawing content parameter acquisition unit 112 determines, as a result of syntax analysis, that the number "90" contained in the character string "90V" following the first specific character string "voltage value" represents the magnitude of the voltage value. In this case, the past drawing content parameter acquisition unit 112 determines that the first content parameter is VN. Similarly, the past drawing content parameter acquisition unit 112 determines that the second content parameter is VN.

Furthermore, as a result of the syntax analysis, the past drawing content parameter acquisition unit 112 determines that the third specific character string "OR" is not followed by a character corresponding to a quantitative variable. In this case, the past drawing content parameter acquisition unit 112 determines that the third content parameter is VL. Similarly, the past drawing content parameter acquisition unit 112 determines that the Lth content parameter is VL.

(Example of processing in the learning preprocessing unit 114)
The learning preprocessing unit 114 acquires content parameter sets corresponding to the drawings a1 to MN (the first to Lth content parameters of each of the drawings a1 to MN) and content parameter variable type information from the past drawing content parameter acquisition unit 112. Then, the learning preprocessing unit 114 generates a content parameter setting table TB3 according to the acquired content parameter variable type information.

The following explanation mainly focuses on the processes based on the content parameter sets (the first to Lth content parameters of drawings a1 to M1, respectively) for drawings a1 to M1 (outline drawings). Therefore, the tables and data for each drawing described below are generated based on the content parameter sets for the outline drawings. Please note that these tables and data are also generated for other types of drawings shown in Figure 2 (e.g., assembly drawings, foundation drawings, and configuration drawings) using a process similar to that described below for outline drawings. Thus, in the reference embodiment, these tables and data are generated for each type of drawing shown in Figure 2.

Figure 4 shows an example of TB3 in its initial state. As shown in Figure 4, TB3 is a table showing the correspondence between (i) the variable type (data type) of each of the first through Lth content parameters and (ii) the preprocessing method to be applied to each of the first through Lth content parameters. The learning preprocessing unit 114 records the acquired content parameter variable type information in the "Variable" field of TB3. Note that the preprocessing method to be applied to each of the first through Lth content parameters has not yet been determined at this stage. For this reason, in the initial state of TB3, all of the "Preprocessing Method" fields are set to blank. In this specification, TB3 in its initial state is referred to as TB3init.

Next, the learning preprocessing unit 114 applies a plurality of types of preprocessing methods to the k-th content parameter according to the k-th content parameter variable type information. Specifically, in the reference embodiment, when the k-th content parameter is VL, the learning preprocessing unit 114 applies the following (i) to (iv) to the k-th parameter:
(i) Processing using raw values as they are (identity processing) (hereinafter referred to as [R]);
(ii) One hot encoding (hereinafter referred to as [O])
;
(iii) Normalization (hereinafter referred to as [N]);
(iii) Standardization (hereinafter referred to as [S]);
Four types of preprocessing methods are applied:

When the kth content parameter is VL, the kth content parameter can also be understood as a value derived by label encoding (hereinafter referred to as [L]) of the kth specific character string by the past drawing content parameter acquisition unit 112. Therefore, when the kth content parameter is VL, [R] can be read as [L]. In this way, when the kth content parameter is VL, [R] and [L] are equivalent preprocessing methods. An example of the preprocessing method [L] will be described later.

On the other hand, when the k-th content parameter is VN, the learning preprocessing unit 114 performs the following (i) to (v) on the k-th parameter:
(i) [R];
(ii) [L];
(iii) [O];
(iv) [N];
(v) [S];
However, as will be apparent to those skilled in the art, the preprocessing technique according to one embodiment of the present invention is not limited to these examples. The preprocessing technique according to one embodiment of the present invention may be any preprocessing technique applicable to VL or VN.

As described above, the learning preprocessing unit 114 expands (data augmentes) the content parameter set by applying multiple preprocessing methods to each content parameter included in the content parameter set according to the kth content parameter variable type information. Hereinafter, the number of VLs and VNs included in the content parameter set will be referred to as FL and FN, respectively.

As is clear from the above description, in the example of the reference embodiment, the total number of combinations of preprocessing patterns for the content parameter set is 4 ^FL × 5 ^FN . Therefore, the learning preprocessing unit 114 expands one content parameter set into 4 ^FL × 5 ^FN content parameter sets. Hereinafter, these 4 ^FL × 5 FN content parameter sets will be collectively referred to as expanded content parameter sets. Each of these 4 ^FL × 5 ^FN content parameter sets ^will be referred to as Dataset 1, Dataset 2, ..., Dataset 4 ^FL × 5 ^FN . The expanded content parameter sets may also be referred to as preprocessed content parameter sets. Note that, for example, Dataset 1 may also be referred to as Dataset No. 1.

The past drawing content parameter set may also be referred to as the search target drawing content parameter set. Therefore, the expanded content parameter set may also be referred to as the past drawing expanded content parameter set (or the search target drawing expanded content parameter set). Similarly, the pre-processed content parameter set may also be referred to as the pre-processed past drawing content parameter set (or the pre-processed search target drawing content parameter set). Each piece of data included in the pre-processed content parameter set may also be referred to as a pre-processed content parameter (more specifically, a pre-processed search target drawing content parameter).

Figure 5 shows an example of the number of post-preprocessing content parameters in each dataset (hereinafter referred to as the number of post-preprocessing content parameters). The number of post-preprocessing content parameters in one dataset (e.g., dataset 1) can also be expressed as the number of data contained in that dataset. Therefore, for example, the number of post-preprocessing content parameters in dataset 1 may be referred to as the number of dimensions (number of elements) of dataset 1. Below, the kth content parameter after preprocessing will be referred to as the kth post-preprocessing content parameter.

As is clear from Figure 6 below, the number of preprocessed content parameters can vary depending on the preprocessing method applied to each content parameter included in the content parameter set. For example, the more content parameters to which [O] is applied, the more likely the number of preprocessed content parameters will increase (see also Figure 8 below).

Figure 6 is a diagram that schematically illustrates the data structure of Dataset 1. In the example of Figure 6, Dataset 1 is a data structure (e.g., a data frame) generated by applying [S] to (i) the first content parameter, (ii) [S] to the second content parameter, (iii) [O] to the third content parameter, and (iv) [L] to the Lth content parameter (see also Figure 8, described below).

In the example of Figure 6, the third content parameter (the content parameter corresponding to the third specific string "OR") is one-hot encoded to generate four pre-processed content parameters corresponding to the third content parameter: "OR_0", "OR_1", "OR_2", and "OR_3".

As an example for ease of explanation, consider a case where the minimum value of the third content parameter (A3) included in the content parameter set is 0 and the maximum value is 3. In other words, consider a case where A3 takes on four discrete values from 0 to 3 in the content parameter set. In this case, each value of A3 can be represented by a four-dimensional one-hot vector (more specifically, a four-bit one-hot vector).

For example, if A3=0,
OR_0=(1,0,0,0);
OR_1=(0,0,0,0);
OR_2=(0,0,0,0);
OR_3=(0,0,0,0);
is.

Also, if A3=1,
OR_0=(0,0,0,0);
OR_1=(0,1,0,0);
OR_2=(0,0,0,0);
OR_3=(0,0,0,0);
is.

Also, if A3=2,
OR_0=(0,0,0,0);
OR_1=(0,0,0,0);
OR_2=(0,0,1,0);
OR_3=(0,0,0,0);
is.

Also, if A3=3,
OR_0=(0,0,0,0);
OR_1=(0,0,0,0);
OR_2=(0,0,0,0);
OR_3=(0,0,0,1);
As described above, A3=i (i in this description is any natural number from 0 to 3) is expressed by a one-hot vector OR_i having a component "1" only in the i-th element.

Next, an example of preprocessing method [N] will be described. Below, we will explain an example in which the learning preprocessing unit 114 normalizes the k-th content parameter (Ak). First, the learning preprocessing unit 114 obtains the maximum value (Akmax) and minimum value (Akmin) of Ak from the content parameter set.

Then, the learning preprocessing unit 114 performs the following:
Ak_Normalized=(Ak-Akmin)/(Akmax-Akmin)
…(1)
Ak_Normalized is calculated as follows. Ak_Normalized is referred to as the normalized kth content parameter (more specifically, the normalized kth past drawing content parameter). The normalized first to Lth content parameters are collectively referred to as the normalized content parameters (more specifically, the normalized past drawing content parameters). The normalized content parameters are an example of preprocessed content parameters.

As described above, the learning preprocessing unit 114 normalizes Ak to Ak_Normalized according to equation (1). Normalization is also called min-max scaling. As can be seen from equation (1), in the dataset generated by [N], the minimum value of the content parameter after normalization is 0, and the maximum value is 1.

Next, an example of the preprocessing method [S] will be described. Below, we will explain an example in which the learning preprocessing unit 114 standardizes Ak. First, the learning preprocessing unit 114 derives the mean value (Akmean) and standard deviation (Aksd) of Ak from the content parameter set.

Next, the learning preprocessing unit 114 performs the following:
Ak_Standardized is calculated as follows: Ak_Standardized = (Ak - Akmean) / Aksd (2). Ak_Standardized is referred to as the standardized kth content parameter (more specifically, the standardized kth past drawing content parameter). The standardized first to Lth content parameters are collectively referred to as the standardized content parameters (more specifically, the standardized past drawing content parameters). The standardized content parameters are an example of preprocessed content parameters.

As described above, the learning preprocessing unit 114 standardizes Ak to Ak_Standardized according to equation (2). As can be seen from equation (2), in the dataset generated by [S], the mean value of the standardized content parameters is 0 and the standard deviation is 1.

The learning preprocessing unit 114 may generate a table (standardization data table) that records the Akmax and Akmin obtained in the [S] process. Similarly, the learning preprocessing unit 114 may generate a table (normalization data table) that records the Akmean and Aksd obtained in the [N] process.

Normalization data table 700A and standardization data table 700B in Figure 7 are examples of a normalization data table and a standardization data table, respectively, in the reference form. Specifically, normalization data table 700A and standardization data table 700B are tables generated in conjunction with the generation of dataset 1.

As can be understood from the above explanation, Akmax and Akmin are not obtained for the kth content parameter to which [N] is not applied. For this reason, in the normalization data table 700A, dummy values (e.g., 0) are assigned to Akmax and Akmin for the kth content parameter to which [N] is not applied.

In the above example, the first to third post-preprocessing content parameters and the Lth post-preprocessing content parameter in dataset 1 were all derived using a preprocessing method other than [N]. Therefore, as shown in Figure 7, in normalization data table 700A, the dummy value 0 is recorded as Akmax and Akmin for the first to third post-preprocessing content parameters and the Nth post-preprocessing content parameter.

In the reference embodiment, the learning preprocessing unit 114 obtains A1mean = 71.9, A1sd = 10.5, A2mean = 2.4, and A2sd = 0.6. In this case, the learning preprocessing unit 114 records these values in the first content parameter and second content parameter fields of the standardization data table 700B.

Note that for kth content parameters that are not subjected to [S], Akmean and Aksd are not obtained in the first place. For this reason, in standardization data table 700B, for kth content parameters that are not subjected to [S], dummy values (e.g., 0) are assigned to Akmean and Aksd. In the example of Figure 7, both the post-preprocessing third content parameter and the preprocessing Lth content parameter are derived using a preprocessing method other than [S]. For this reason, in standardization data table 700B, the dummy value 0 is recorded as Akmax and Akmin for the post-preprocessing third content parameter and the preprocessing Lth content parameter.

The learning preprocessing unit 114 may generate a table (dataset/preprocessing method correspondence table) that shows the correspondence between each dataset and each preprocessing method. Table TB4 in Figure 8 is an example of a dataset/preprocessing method correspondence table. Note that the reason why the preprocessing method corresponding to dataset 1 in the example in Figure 8 is hatched will be explained later.

9 illustrates some of the multiple data sets generated by the learning preprocessing unit 114. In FIG. 9, reference numeral 900A represents data set 1, reference numeral 900B represents data set 2, and reference numeral 900C represents data set 4 ^FL × 5 ^FN . As can be understood from the above description, the first and second preprocessed content parameters in data set 1 are the first and second standardized content parameters, respectively. The first and second standardized content parameters are values obtained by standardizing the first and second content parameters according to the above-described formula (2).

As can be seen from Figure 8 above, Dataset 2 in the example of Figure 9 differs from Dataset 1 in that it is generated by applying [L] to the third content parameter. The preprocessing methods for the other content parameters are the same as those used in the example of Dataset 1.

As described above, unlike Dataset 1, Dataset 2 does not apply [O] to the third content parameter. Therefore, the number of dimensions of Dataset 2 is smaller than the number of dimensions of Dataset 1. Specifically, as shown in Figure 5 above, the number of dimensions of Dataset 1 is 50, and the number of dimensions of Dataset 2 is 28.

8 described above, the dataset 4 ^FL × 5 ^FN in the example of FIG. 9 is generated by applying [O] to all content parameters, unlike datasets 1 and 2. Therefore, the number of dimensions of dataset 4 ^FL × 5 ^FN is greater than the number of dimensions of datasets 1 and 2. Specifically, as shown in FIG. 5, the number of dimensions of dataset 4 ^FL × 5 ^FN is 151.

(Learning Phase in the Learning Model Generation Unit 113s)
The processing in the learning model generation unit 113s can be broadly divided into a learning phase and a verification phase. First, the learning phase will be described. The learning model generation unit 113s acquires an expanded content parameter set (dataset 1 to dataset 4 ^FL × 5 ^FN ) from the learning preprocessing unit 114. Then, the learning model generation unit 113s divides each of data sets 1 to 4 ^FL × 5 ^FN into training data and verification data.

The learning model generation unit 113s generates multiple learning models by applying each of multiple predetermined types of machine learning algorithms to any one of the datasets 1 to 4 ^FL × 5 ^FN (for convenience, referred to as a dataset of interest). Specifically, the learning model generation unit 113s generates multiple learning models using the training data of the dataset of interest by applying each of multiple predetermined types of machine learning algorithms.

As an example, the learning model generation unit 113s generates multiple learning models corresponding to Dataset 1 using the training data of Dataset 1 by applying each of multiple predetermined types of machine learning algorithms. In this way, the learning model generation unit 113s generates multiple learning models corresponding to the dataset of interest.

In the example of the reference form, the multiple types of machine learning algorithms include a gradient-based machine learning algorithm and a distance-based machine learning algorithm. A gradient-based machine learning algorithm is a machine learning algorithm that uses gradient descent. Specific examples of gradient-based machine learning algorithms include DT (Decision Tree), LR (Logistic Regression), and NN (Neural Network). The above-mentioned multinomial logistic regression is an example of RS.

The DT in one aspect of the present invention, more strictly speaking, refers to a GBDT (Gradient Boosting DT, gradient boosting decision tree). Therefore, examples of the DT in one aspect of the present invention include XGBoost (eXtreme Gradient Boosting) and LightGBM (Light Gradient Boosting Machine).

A distance-based machine learning algorithm is a machine learning algorithm that evaluates (e.g., classifies) each piece of input data based on the distance between each piece of input data in a metric space that represents the distribution of one or more parameters contained in each piece of input data. Specific examples of distance-based machine learning algorithms include SVM (Support Vector Machine) and multiple regression.

As described above, the learning model generation unit 113s may generate multiple learning models by exhaustively applying each machine learning algorithm to each of the data sets 1 to 4 ^FL × 5 ^FN , thereby generating a sufficient number of learning models to be evaluated (verified) in the verification phase described below.

In addition, the learning model generation unit 113s may change each hyperparameter set of a certain machine learning algorithm. In this case, the learning model generation unit 113s uses the changed hyperparameter set to apply the same machine learning algorithm and generate a further learning model. In this way, the learning model generation unit 113s may generate multiple learning models by comprehensively applying hyperparameter sets as well. This allows for the generation of even more learning models (see Figure 10, described below).

(Verification phase in the learning model generation unit 113s)
Next, the verification phase will be described. The learning model generation unit 113s evaluates the quality of each of the multiple learning models generated in the learning phase using each of the data sets 1 to 4 ^FL × 5 ^FN (more specifically, using the verification data for each of the data sets 1 to 4 ^FL × 5 ^FN ).

As an example, consider dataset j as the dataset of interest. j is a subscript indicating the column number of TB5 (see Figure 10), which will be described later. The learning model generation unit 113s uses the validation data of dataset j to obtain an index value indicating the prediction accuracy (determination accuracy) of each of the multiple learning models corresponding to dataset j. For example, the learning model generation unit 113 inputs the validation data of dataset j into model (i, j), which will be described later, to output (derive) the index value to model (i, j). The index value can also be expressed as an index value indicating the quality of model (i, j).

In the reference embodiment, the learning model generation unit 113s acquires Accuracy (the rate of correct answers) as the index value. Therefore, the prediction accuracy in the reference embodiment may also be referred to as search accuracy (more specifically, search accuracy of past drawings) (see Figure 10). However, as will be clear to those skilled in the art, the index value according to one aspect of the present invention is not limited to the above example, and other index values known in the field of machine learning may also be used. Therefore, for example, the learning model generation unit 113s may acquire Precision (the precision rate) or Recall (the recall rate) as the index value. Alternatively, the learning model generation unit 113s may acquire the F-score as the index value. As is well known, the F-score is the harmonic mean of Precision and Recall.

The learning model generation unit 113s selects the best learning model based on the multiple index values obtained. In the reference example, the learning model generation unit 113s identifies the maximum value (maximum index value) among the multiple index values. Then, the learning model generation unit 113s selects the learning model with the maximum index value as the best learning model.

The learning model generation unit 113s may generate a table (evaluation result table) that shows the evaluation results from the evaluation phase. Table TB5 in Figure 10 is an example of an evaluation result table. In TB5, one index value is recorded so that there is a one-to-one correspondence between one dataset, one machine learning algorithm, and one hyperparameter set (e.g., Para1).

In the example of Fig. 10, Para1 and Para2 each represent a hyperparameter set (a series of hyperparameters) applied to a certain machine learning algorithm (e.g., DT).
・DT Para 1: Data division method = "gni", maximum depth = 3, ...
・DT Para 2: Data division method = "entropy", maximum depth = 3, ...
・LR Para1: Regularization type = "l2", regularization term coefficient = 1.0, ...
・LR Para2: Regularization type = "l2", regularization coefficient = 0.5, ...
・NN Para1: batch size = 256, maximum number of learning iterations = 1000, ...
NN Para2: batch size = 128, maximum number of learning iterations = 1000, ...
・SVM Para1: Kernel type = "rbf", regularization coefficient = 1.0, ...
・SVM Para2: Kernel type = "rbf", regularization coefficient = 0.5, ...
As will be apparent to those skilled in the art, the number of hyperparameter sets is not limited to two. For example, five hyperparameter sets, Para1 to Para5, may be assigned to each machine learning algorithm.

In the following, the component in the i-th row and j-th column of TB5 will be denoted as TB5(i,j). The learning model corresponding to TB5(i,j) will be referred to as model(i,j). In TB8, machine learning algorithms and their hyperparameter sets are arranged in the row direction (i direction). Datasets are arranged in the column direction (j direction). Therefore, as an example, TB8(1,1) in Figure 10 is the search accuracy obtained when Para1 is applied in DT. In the example of Figure 10, TB5(i,j) = 42.1%. As mentioned above, TB5(1,1) can also be considered an index value indicating the quality of the learning model generated by DT to which Para1 is applied, i.e., model(1,1).

For ease of explanation, in the example in Figure 10, the row number corresponding to Para1 in the NN will be denoted as im. In the example in Figure 10, TB5(im,1) = 81.6% is the maximum value among each TB5(i,j) (see the hatched cells in TB5).

As described above, the learning device 11s generates multiple learning models based on the content parameter set (for convenience, referred to as the first drawing type content parameter set) for drawings a1 to M1 (outline drawings). The learning device 11 then evaluates the quality of each of the multiple learning models generated based on the first drawing type content parameter set (more specifically, derives index values for each of the multiple generated learning models).

Similar to the example above for the outline drawing, the learning device 11s generates multiple learning models for each type of drawing based on the content parameter set for that drawing. The learning device 11s then evaluates the quality of each of the multiple learning models generated based on the content parameter set.

As one example, the learning device 11s generates multiple learning models based on a content parameter set (for convenience, referred to as the second content drawing type parameter set) for drawings a2 to M2 (assembly drawings). The learning device 11 then evaluates the quality of each of the multiple learning models generated based on the second drawing type content parameter set. As another example, the learning device 11s generates multiple learning models based on a content parameter set (for convenience, referred to as the Nth drawing type content parameter set) for drawings aN to MN (configuration drawings). The learning device 11s then evaluates the quality of each of the multiple learning models generated based on the Nth drawing type content parameter set.

In the reference embodiment, as described above, of all the index values derived based on the first drawing type content parameter set through the Nth drawing type content parameter set, TB5(im,1) is assumed to be the maximum value. Therefore, in the reference embodiment, the learning model generation unit 113s identifies TB5(im,1) as the maximum index value. The learning model generation unit 113s then selects the learning model with the maximum index value, i.e., model (im,1), as the best learning model. As described above, in the example of the reference embodiment, the learning model generation unit 113s selects the highest quality learning model as the best learning model from among the multiple learning models generated in the learning phase. Note that in this specification, the machine learning algorithm corresponding to the best learning model is referred to as the best machine learning algorithm. In the example of Figure 10, the best machine learning algorithm is NN.

As will be clear to those skilled in the art, the method for selecting the best learning model is not limited to the above example. The learning model generation unit 113s may simply select the best learning model from among multiple learning models based on multiple index values. For example, the learning model generation unit 113s may derive a statistical value based on multiple index values and select the best learning model based on the statistical value. The maximum index value in the reference embodiment is an example of a statistical value.

(Additional information about the learning phase)
However, unlike gradient-based machine learning algorithms, distance-based machine learning algorithms are known to be affected by the so-called "curse of dimensionality." Therefore, when a dataset of interest has a high number of dimensions, a learning model generated by a distance-based machine learning algorithm (hereinafter referred to as a distance-based learning model) tends to be of lower quality than a learning model generated by a gradient-based machine learning algorithm (hereinafter referred to as a gradient-based learning model). For this reason, when a dataset of interest has a high number of dimensions, it is considered unlikely that a distance-based learning model generated using the dataset of interest will be selected as the best learning model.

Therefore, in a reference embodiment, it is preferable that the learning model generation unit 113 stop generating a learning model using a distance-based machine learning algorithm with a data set of interest when the number of dimensions of the data set of interest is equal to or greater than a predetermined dimensionality threshold Dth. This makes it possible to prevent the generation of a learning model that is expected to be of low quality, thereby reducing the computational cost in the learning phase. In addition, it is also possible to reduce the computational cost in the subsequent evaluation phase.

In the field of machine learning, it is empirically known that when the number of dimensions of a dataset is 30 or more, the quality of distance-based learning models tends to decline. Therefore, for example, Dth may be set to a predetermined value of 30 or more. In the reference embodiment, a case where Dth is set to 30 is illustrated.

As described above, in the example of the reference embodiment, the number of dimensions of Dataset 1 is 50, and the number of dimensions of Dataset 4 ^FL × 5 ^FN is 151. Therefore, in the example of Fig. 10, the learning model generation unit 113s does not generate learning models using a distance-based machine learning algorithm (e.g., SVM) for Dataset 1 and Dataset 4 ^FL × 5 ^FN (see cells marked with "x" in TB5).

10, the learning model generation unit 113s generates learning models by applying only gradient-based machine learning algorithms (e.g., DT, LR, and NN) to Data Set 1 and Data Set 4 ^FL × 5 ^FN . In this way, the learning model generation unit 113s generates only gradient-based learning models for Data Set 1 and Data Set 4 ^FL × 5 ^FN , without generating distance-based learning models.

On the other hand, if the number of dimensions of the dataset of interest is less than Dth, the learning model generation unit 113s may apply a distance-based machine learning algorithm to generate a learning model using the dataset. In the example of the reference form, the number of dimensions of dataset 2 is 28. Therefore, in the example of Figure 10, the learning model generation unit 113 generates a gradient-based learning model for dataset 2, and also generates a distance-based learning model.

(Processing after the verification phase in the learning model generation unit 113s)
The learning model generation unit 113s selects, as the best dataset, one dataset corresponding to the best learning model from among the datasets included in TB5. In the example of Fig. 10, the learning model generation unit 113s selects Dataset 1 as the best dataset.

The learning model generation unit 113s then selects the preprocessing method corresponding to the best data set as the best preprocessing method. In the reference example, the learning model generation unit 113s references the above-mentioned TB4 and reads out the preprocessing method corresponding to Data Set 1 as the best preprocessing method (see the hatched area in Figure 8). As can be understood from the above explanation, the learning model generation unit 113s selects the preprocessing method corresponding to the best learning model as the best preprocessing method.

The learning model generation unit 113s then updates TB3init by recording the best preprocessing method in the "Preprocessing Method" field in the above-mentioned TB3init. In this specification, the updated content parameter setting initial table is referred to as TB3new. Figure 11 shows an example of TB3new. In the example of Figure 11, the preprocessing method corresponding to Dataset 1 (in other words, the preprocessing method corresponding to the best learning model) is recorded as the best preprocessing method in the "Preprocessing Method" field.

(Drawing search device 12)
Next, a description will be given of the drawing search device 12. The drawing search device 12 includes a new drawing data acquisition unit 121, a new drawing content parameter acquisition unit 122 (target drawing content parameter acquisition unit), a search preprocessing unit 125, and a search unit 126.

The drawing search device 12 searches for at least one drawing by matching the target drawing with each of the multiple search target drawings using the learning model (in the example of the reference embodiment, the best learning model) generated by the learning device 11s. Drawing ND in the reference embodiment is an example of a target drawing. As described below, the drawing search device 12 uses the learning model to match drawing ND with each of drawings a1 to MN.

(Acquisition of new drawings)
The new drawing data acquisition unit 121 is a functional unit paired with the past drawing data acquisition unit 111. As an example, the new drawing data acquisition unit 121 acquires a predetermined new drawing (e.g., drawing ND) included in the new property data set of the new property drawing DB 92 when the input unit 71 receives a predetermined user operation. The new drawing data acquisition unit 121 supplies the acquired drawing ND to the new drawing content parameter acquisition unit 122.

(Getting the content parameter set corresponding to the new drawing)
The new drawing content parameter acquisition unit 122 is a functional unit paired with the previous drawing content parameter acquisition unit 112. The new drawing content parameter acquisition unit 122 acquires a content parameter set corresponding to the drawing ND by the same processing as the previous drawing content parameter acquisition unit 112. That is, the new drawing content parameter acquisition unit 122 acquires the content parameters of the drawing ND by analyzing the drawing ND by the same analysis method as the previous drawing content parameter acquisition unit 112.

Hereinafter, the kth content parameter of drawing ND will also be referred to as Ck. Note that, to distinguish it from the search target drawing content parameters described above, the content parameter of the target drawing (drawing ND) will also be referred to as the target drawing content parameter. The kth content parameter of the target drawing will also be referred to as the target drawing kth content parameter. The target drawing content parameter may also be referred to as the new drawing content parameter. Therefore, the target drawing kth content parameter may also be referred to as the new drawing kth content parameter.

As described above, the new drawing content parameter acquisition unit 122 sets C1 to CL by performing the same processing on the drawing ND as the past drawing content parameter acquisition unit 112. The new drawing content parameter acquisition unit 122 may then generate a new drawing content parameter table TB-ND that indicates C1 to CL. Figure 12 shows an example of TB-ND.

(Example of processing in search preprocessing unit 125)
The search preprocessing unit 125 acquires a content parameter set (for convenience, referred to as a new drawing content parameter set) corresponding to the drawing ND from the new drawing content parameter acquisition unit 122. Specifically, the new drawing content parameter set refers to a data set including the first to Lth content parameters (C1 to CL) of the drawing ND. As an example, the search preprocessing unit 125 acquires the above-mentioned TB-ND from the new drawing content parameter acquisition unit 122.

The search preprocessing unit 125 also obtains the best preprocessing method from the learning model generation unit 113s. As an example, the search preprocessing unit 125 obtains TB3 from the learning model generation unit 113s and reads the best preprocessing method from TB3.

Next, the search preprocessing unit 125 generates a preprocessed new drawing content parameter set by preprocessing the new drawing content parameter set according to the best preprocessing method. That is, the search preprocessing unit 125 generates a preprocessed new drawing content parameter set by applying the same preprocessing method as the best preprocessing method to each of C1 to CL. In the example of the reference embodiment, the search preprocessing unit 125 (i) applies [S] to C1, (ii) applies [S] to C2, (iii) applies [O] to C3, and (iv) applies [L] to CL.

Note that the new drawing content parameter set may also be referred to as the target drawing content parameter set. Therefore, the pre-processed new drawing content parameter set may also be referred to as the pre-processed target drawing content parameter set.

The search preprocessing unit 125 may generate a table indicating the post-preprocessing new drawing content parameter set (post-preprocessing new drawing content parameter table). TB-NDP shown in Figure 13 is an example of a post-preprocessing new drawing content parameter table in the reference form. As is clear from the above explanation, the post-preprocessing new drawing content parameter set has the same data structure as Dataset 1 (see also Figure 6 above).

An example of the preprocessing method [S] in the search preprocessing unit 125 is as follows. The following describes the case where Ck (the kth content parameter of the drawing ND) is standardized by the search preprocessing unit 125.

First, the search preprocessing unit 125 references the standardization data table 700B and obtains Akmean and Aksd.
Ck_Standardized=(Ck-Akmean)/Aksd...(3)
Ck_Standardized is calculated as follows. Ck_Standardized is the standardized k-th content parameter in the drawing ND. Ck_Standardized is also referred to as the standardized new drawing k-th content parameter. The standardized new drawing k-th content parameter is an example of the preprocessed new drawing k-th content parameter.

An example of the preprocessing method [N] in the search preprocessing unit 125 will be described as follows. First, the search preprocessing unit 125 references the normalization data table 700A described above to obtain Akmax and Akmin. Next, the search preprocessing unit 125
Ck_Normalized=(Ck-Akmin)/(Akmax-Akmin)
…(4)
Ck_Normalized is calculated as follows. Ck_Normalized is the normalized k-th content parameter in the drawing ND. Ck_Normalized is also referred to as the normalized new drawing k-th content parameter. The normalized new drawing k-th content parameter is another example of the preprocessed new drawing k-th content parameter.

(Search Phase in Search Unit 126)
The search unit 126 acquires the post-preprocessing new drawing content parameter set from the search preprocessing unit 125. The search unit 126 also acquires the best learning model from the learning model generation unit 113. The search unit 126 inputs the post-preprocessing new drawing content parameter set to the best learning model. Then, the search unit 126 acquires the output of the best learning model corresponding to the post-preprocessing new drawing content parameter set from the best learning model.

As an example, consider a case where each learning model in the reference embodiment is a learning model trained to output (derive) a relevance score, which is a score (index) indicating the degree of relevance of each past drawing (drawings a1 to MN) to drawing ND. For an example of a method for deriving a relevance score, see Patent Document 1.

In this case, the search unit 126 inputs the preprocessed new drawing content parameter set into the best learning model, causing the best learning model to output a relevance score corresponding to the preprocessed new drawing content parameter set. Then, the search unit 126 searches for at least one past drawing corresponding to the drawing ND based on the output (e.g., relevance score) of the best learning model. For an example of a method for searching for past drawings based on a relevance score, see Patent Document 1. As with Patent Document 1, the search unit 126 may display data indicating the search results on the display unit 72.

(Effects of the reference form)
The information processing system 100s (information processing device 1s) in the reference embodiment, like the prior art (the technology of Patent Document 1), can improve user convenience in drawing searches compared to conventional methods. In addition, unlike the prior art, the learning device 11s comprehensively applies multiple types of preprocessing methods to a search target drawing content parameter set (past drawing content parameter set), thereby expanding the search target drawing content parameter set. In other words, multiple preprocessed search target drawing content parameter sets are generated.

Next, by applying multiple machine learning algorithms, multiple learning models are generated using multiple preprocessed retrieval target drawing content parameter sets (e.g., data sets 1-4 ^FL × 5 ^FN ). Then, the best learning model is selected from the multiple learning models based on an index value indicating the quality of each of the multiple learning models (e.g., retrieval accuracy of past drawings). In other words, the best machine learning algorithm is selected from the multiple machine learning algorithms based on the index value. Then, the best preprocessing method corresponding to the best learning model is selected.

In general, the quality of a learning model generated by a machine learning algorithm may vary depending on the preprocessing method applied to the learning data (e.g., the set of drawing content parameters to be searched). In addition, the quality of the learning model may also vary depending on the type of machine learning algorithm applied to the preprocessed learning data.

When appropriate preprocessing methods and machine learning algorithms are selected, it is possible to generate high-quality learning models. However, there are a wide variety of preprocessing methods and machine learning algorithms that can be applied in the field of machine learning. For this reason, it is not always easy for users to manually select a combination of preprocessing methods and machine learning algorithms that is suitable (ideally the optimal) for improving the quality of learning models.

Therefore, as described above, the learning device 11s comprehensively evaluates each of the multiple generated learning models based on the above index values to select the best learning model and the best preprocessing method. In other words, the best learning model and the best preprocessing method are selected by performing a grid search on the multiple generated learning models.

The above configuration makes it possible to identify the best learning model and best preprocessing method without the user having to manually select them. In other words, the learning device 11 can automatically select a combination of preprocessing method and machine learning algorithm that is expected to be optimal (or at least suitable) for improving the quality of the learning model.

Then, the drawing search device 12 applies preprocessing to the target drawing content parameter set (new drawing content parameter set) according to the best preprocessing method selected by the learning device 11s. In other words, a preprocessed target drawing content parameter set (preprocessed new drawing content parameter set) is generated according to the best preprocessing method.

With the above configuration, a preprocessed target drawing content parameter set is generated as an input dataset having a data structure suitable for the best learning model. Therefore, by inputting this preprocessed target drawing content parameter set into the best learning model, it is possible to obtain a learning model output (e.g., a relevance score derived by the best learning model) with even higher accuracy than the prior art described above.

As described above, the information processing system 100s can cause the drawing search device 12 to perform a search using (i) the best preprocessing method pre-selected by the learning device 11s and (ii) the best learning model pre-generated by the learning device 11s. As a result, the information processing system 100s can achieve even higher search accuracy than the prior art.

(Supplementary information in reference format)
An example of the preprocessing method [L] will be described below. In a reference embodiment, the learning preprocessing unit 114 may label-encode raw values (content parameters acquired by the past drawing content parameter acquisition unit 112) according to a raw value-label value conversion table. Specifically, the learning preprocessing unit 114 may convert raw values into label values according to a table (hereinafter referred to as the raw value-label value conversion table) that indicates the correspondence between a certain raw value and the label value corresponding to the raw value.

As an example, in the information processing system 100s, individual raw value-to-label value conversion tables are pre-set for each of the first through Lth content parameters. Hereinafter, the raw value-to-label value conversion table corresponding to the kth content parameter will be referred to as the kth raw value-to-label value conversion table.

Figure 14 illustrates some of the multiple raw value-to-label value conversion tables. In Figure 14, (i) symbol 1400-1 represents the first raw value-to-label value conversion table, (ii) symbol 1400-2 represents the second raw value-to-label value conversion table, (iii) symbol 1400-3 represents the third raw value-to-label value conversion table, and (iv) symbol 1400-L represents the Lth raw value-to-label value conversion table.

In the example of FIG. 14, the learning preprocessing unit 114 may label encode the kth content parameter according to the kth raw value-to-label value conversion table. For example, the learning preprocessing unit 114 label encodes the first content parameter according to the first raw value-to-label value conversion table. Also, the learning preprocessing unit 114 label encodes the Lth content parameter according to the Lth raw value-to-label value conversion table.

As described above, regardless of the variable type of the kth content parameter (i.e., whether the kth content parameter is VL or VN), the preprocessing method [L] may be applied to the kth content parameter. Note that, as mentioned above, when the kth content parameter is VL, [L] can be said to be a preprocessing method equivalent to [R].

As another example, the information processing system 100s may have created in advance a table that combines the first to Lth raw value-label value conversion tables (hereinafter referred to as the integrated raw value-label value conversion table). Reference numeral 1500 in Figure 15 represents an example of an integrated raw value-label value conversion table.

In the example of Figure 15, the kth row of the integrated raw value/label value conversion table corresponds to the kth raw value/label value conversion table. Therefore, the learning preprocessing unit 114 may refer to the kth row of the integrated raw value/label value conversion table and label encode the kth content parameter. For example, the learning preprocessing unit 114 refers to the second row of the integrated raw value/label value conversion table and label encodes the second content parameter.

[Embodiment 1]
16 is a block diagram showing the configuration of the main parts of the information processing system 100 of embodiment 1. The information processing device of the information processing system 100 is referred to as the information processing device 1. The control device of the information processing device 1 is referred to as the control device 10. The learning device of the control device 10 is referred to as the learning device 11 (model generation device).

Unlike learning device 11s, learning device 11 further includes a determination unit 115. Furthermore, learning device 11 includes a learning model generation unit 113 (learning unit) instead of the learning model generation unit 113s of learning device 11s. Before explaining the operation of each unit of learning device 11, possible improvements in the reference embodiment will be described below.

As described above, in the reference embodiment, the content parameter set is expanded by applying multiple preprocessing methods to the kth content parameter according to the kth content parameter variable type information. Next, the best learning model and best preprocessing method are selected through the generation and verification (evaluation) of a learning model using the expanded content parameter set.

However, in the reference embodiment, depending on the preprocessing method applied to the kth content parameter, multicollinearity may occur between the multiple explanatory variables included in the expanded content parameter set (between the multiple preprocessed content parameters). As will be understood by those skilled in the art, if a learning model is generated using multiple explanatory variables in which multicollinearity occurs, the quality of the learning model may deteriorate.

Furthermore, as will be understood by those skilled in the art, the risk of multicollinearity increases as the number of dimensions of the training data used to generate a certain training model (in the example of the reference embodiment, the number of post-preprocessing content parameters) increases. As described above, one of the multiple types of preprocessing methods can be [O] (one-hot encoding). As can be understood from the explanation in the reference embodiment, [O] can be said to be a typical example of a preprocessing method that results in an increase in the number of post-preprocessing content parameters.

Therefore, there is concern that the risk of multicollinearity increasing as [O] is applied to more k-th content parameters. Therefore, in order to further improve the quality of the learning model (e.g., to obtain a higher quality best learning model), it is considered preferable to introduce measures to eliminate multicollinearity. The learning device 11 of embodiment 1 was newly created by the inventors of the present application based on this idea.

(Example of processing by the determination unit 115)
The following describes a series of processes performed by the determination unit 115 for one data set obtained by expanding the content parameter set (i.e., the above-mentioned data set of interest). The following describes a case where data set 1 indicated by reference numeral 900A in FIG. 9 is the data set of interest. The determination unit 115 calculates a coefficient of determination for each of two different post-preprocessing content parameter sets included in data set 1.

Specifically, the determination unit 115
…(5)
The coefficient of determination Rij is calculated as follows. The coefficient of determination Rij is the coefficient of determination between Ik and Jk described below. In the description of the first embodiment, i is an integer that satisfies 1≦i<P, and j is an integer that satisfies i<j≦P. P is the number of dimensions (number of rows) of the dataset of interest. P may also be referred to as the number of series of the dataset of interest. In the example of the first embodiment, P=50 (see FIG. 5 above).

In equation (5), Ik is the i-th preprocessed content parameter (one preprocessed content parameter) of the P preprocessed content parameters included in dataset 1. Jk is the j-th preprocessed content parameter (another preprocessed content parameter) of the P preprocessed content parameters included in dataset 1. Q is the number of items (number of columns) in the dataset of interest. In the example of embodiment 1, Q = M. Iave is the average value of I1 to IQ. Jave is the average value of J1 to JQ.

As described above, the determination unit 115 may calculate the coefficient of determination between any two different preprocessed content parameters included in the target data set. In the first embodiment, the determination unit 115 calculates _{two P} C combinations of Rij according to formula (5). That is, the determination unit 115 calculates Rij for each combination pattern of two different search target drawing content parameters included in the target data set.

As shown in equation (5), Rij can be expressed as the square of the correlation coefficient sij between Ik and Jk. Therefore, Rij can be used as an evaluation value (multicollinearity evaluation value) that indicates the degree (strength) of multicollinearity between Ik and Jk. Rij in embodiment 1 is an example of a multicollinearity evaluation value.

The determination unit 115 compares each calculated coefficient of determination with a predetermined threshold Rth (coefficient of determination threshold). The coefficient of determination threshold is an example of a threshold for the multicollinearity assessment value (multicollinearity threshold). In this specification, a multicollinearity assessment value that satisfies the condition that "the multicollinearity assessment value is equal to or greater than the multicollinearity threshold" is referred to as a high-risk multicollinearity assessment value. Therefore, in embodiment 1, a coefficient of determination that satisfies the condition that "the coefficient of determination is equal to or greater than Rth" is referred to as a high-risk coefficient of determination.

In embodiment 1, the determination unit 115 compares each coefficient of determination with Rth for each preprocessed content parameter and extracts high-risk coefficients of determination from among the coefficients of determination. The determination unit 115 then counts up the number of high-risk coefficients of determination extracted for each preprocessed content parameter.

In the field of data science, when the absolute value of the correlation coefficient between two explanatory variables (hereinafter also referred to as the absolute value of the correlation coefficient) is 0.7 or more, it is often evaluated that there is a strong correlation (or a somewhat strong correlation) between the two explanatory variables. For this reason, as an example, it is preferable that Rth is set to a predetermined value of 0.49 (= ^0.72 ) or more and 1 or less. In embodiment 1, a case where Rth is set to 0.49 is exemplified. However, as will be clear to those skilled in the art, Rth is not limited to the above example.

The determination unit 115 generates a table (coefficient of determination/number of high-risk coefficients of determination table) that shows the correspondence between each Rij and the number of high-risk coefficients of determination. TB6 in Figure 17 is an example of a coefficient of determination/number of high-risk coefficients of determination table. As described below, TB6 is updated through a series of processes in the determination unit 115. For this reason, in this specification, TB6 in its initial state is also referred to as TB6init.

Figure 17 shows an example of TB6init. The coefficient of determination Rij between the i-th preprocessed content parameter (i.e., Ik) and the j-th preprocessed content parameter (i.e., Jk) is recorded in the i-th row and j-th column of TB6init. The rightmost column (P+1 column) of TB6init records the number of high-risk coefficients of determination corresponding to Ik (for ease of explanation, also referred to as "the number of high-risk coefficients of determination possessed by Ik"). In the example of Figure 17, high-risk coefficients of determination are hatched.

As an example, the determination unit 115 may determine, among the pre-processing content parameters in TB6 (in TB6init in the example of Figure 17), the pre-processing content parameter with the highest number of high-risk determination coefficients (highest number of high-risk determination coefficients) as the pre-processing content parameter to be deleted. In the example of Figure 17, among the pre-processing content parameters included in TB6init, the pre-processing content parameter "OR_2" in the fifth row has the highest number of high-risk determination coefficients (6) (see the dotted rectangle in "Number of high-risk determination coefficients" in Figure 17). Therefore, the determination unit 115 determines the pre-processing content parameter "OR_2" as the pre-processing content parameter to be deleted. Then, the determination unit 115 updates TB6 by deleting the row and column corresponding to "OR_2" (i.e., the fifth row and fifth column) from TB6init.

Table 6a in Figure 18 is an example of Table 6 obtained by updating Table 6init as described above. After deleting the row and column corresponding to "OR_2" from Table 6init, the determination unit 115 updates the number of high-risk determination coefficients corresponding to Ik that was recorded in the rightmost column of Table 6init. Therefore, the rightmost column of Table 6a records the updated number of high-risk determination coefficients.

In the example of Figure 18, of the pre-processed content parameters included in TB6a, the pre-processed content parameter "Current Value" in the second row and the pre-processed content parameter "OR_1" in the fourth row each have the highest number of high-risk determination coefficients (3) (see the dotted rectangle in "Number of High-Risk Determination Coefficients" in Figure 18). In this way, it is possible that multiple pre-processed content parameters in TB6 have the same number of highest number of high-risk determination coefficients.

In such a case, the determination unit 115 may determine the multiple pre-processing content parameters having the largest number of high-risk determination coefficients as the deletion candidate post-processing content parameters. Next, the determination unit 115 may determine the deletion candidate post-processing content parameter having the largest number of determinations among the multiple deletion candidate post-processing content parameters as the deletion candidate post-processing content parameter. Note that the deletion candidate post-processing content parameter may also be referred to as a non-learning candidate post-processing content parameter. Similarly, the deletion candidate post-processing content parameter may also be referred to as a non-learning candidate post-processing content parameter.

In the example of Figure 18, the maximum value of the coefficient of determination for the preprocessed content parameter "current value" in the second row is greater than the maximum value of the coefficient of determination for the preprocessed content parameter "OR_1" in the fourth row. Therefore, the determination unit 115 determines the preprocessed content parameter "current value" in the second row as the preprocessed content parameter to be deleted (see the hatched rectangle in "Number of high-risk determination coefficients" in Figure 18). The determination unit 115 then updates TB6 by deleting the row and column corresponding to "current value" (i.e., the second row and second column) from TB6a.

Table 6b in Figure 19 is an example of table 6 obtained by updating table 6a as described above. After deleting the row and column corresponding to "current value" from table 6a, the determination unit 115 updates the number of high-risk determination coefficients corresponding to Ik, which was recorded in the rightmost column of table 6a. Therefore, the rightmost column of table 6b records the updated number of high-risk determination coefficients. Thereafter, the determination unit 115 repeats updating table 6 as described above until the number of all high-risk determination coefficients in table 6 becomes 0. Note that in the example of Figure 19, the preprocessed content parameter "OR_1" in the third row has been determined as the preprocessed content parameter candidate for deletion (see the dotted rectangle in "Number of high-risk determination coefficients" in Figure 19). However, in the example of Figure 19, a preprocessed content parameter not shown has been determined as the preprocessed content parameter to be deleted.

TB6end in Figure 20 is an example of the final TB6 obtained as a result of the repeated updates described above. As shown in Figure 20, in TB6end, the counts of all high-risk determination coefficients are 0. The preprocessed content parameters included in TB6end are the preprocessed content parameters obtained by excluding each preprocessed content parameter to be deleted (in the range illustrated in the example of embodiment 1) found during the TB6 update process from the multiple preprocessed content parameters included in TB6init. For this reason, the preprocessed content parameters included in TB6end may also be referred to as residual preprocessed content parameters.

As is clear from the above explanation, no high-risk coefficients of determination occur between the multiple residual preprocessed content parameters. In other words, the risk of multicollinearity occurring between the multiple residual preprocessed content parameters is sufficiently reduced. Therefore, the determination unit 115 may process (more specifically, prune) dataset 1 so that only the residual preprocessed content parameters are used as explanatory variables in the learning phase.

Specifically, the determination unit 115 may generate pruned dataset 1 (a pruned dataset corresponding to dataset 1) by deleting the post-preprocessing content parameters to be deleted from dataset 1. Reference numeral 900AP in FIG. 21 represents an example of pruned dataset 1. As is clear from the above explanation, pruned dataset 1 in the example of FIG. 21 is generated by deleting "OR_2" and "current value" from dataset 1.

As described above, the determining unit 115 may generate a pruned dataset of interest corresponding to the dataset of interest by deleting, from the dataset of interest, post-preprocessing content parameters to be deleted in the dataset of interest. Therefore, for example, the determining unit 115 may generate pruned dataset 2 by deleting, from dataset 2, post-preprocessing content parameters to be deleted in dataset 2, in the same manner as in the above example regarding dataset 1. Furthermore, the determining unit 115 may generate pruned dataset 4 ^FL × 5 ^FN by deleting, from dataset 4 ^FL × 5 ^FN , post-preprocessing content parameters to be deleted in dataset 4 ^FL × 5 ^FN .

In this way, the determination unit 115 may generate a pruned post-preprocessing first drawing type content parameter set (a set consisting of pruned dataset 1 to pruned dataset 4 FL x 5 FN) by deleting the preprocessing content parameters to be deleted in the preprocessing first drawing type content parameter set (a ^data set obtained by applying each of multiple patterns of preprocessing to the first drawing type content parameter set) from ^the preprocessing first drawing type content parameter set.

Similar to the above example for the first drawing type content parameter set, the determination unit 115 may generate the pruned post-preprocessing second drawing type content parameter set by deleting from the post-preprocessing second drawing type content parameter set the post-preprocessing content parameters to be deleted in the post-preprocessing second drawing type content parameter set (a data set obtained by applying each of multiple patterns of preprocessing to the second drawing type content parameter set). Furthermore, the determination unit 115 may generate the pruned post-preprocessing Nth drawing type content parameter set by deleting from the preprocessing Nth drawing type content parameter set the post-preprocessing content parameters to be deleted in the preprocessing Nth drawing type content parameter set (a data set obtained by applying each of multiple patterns of preprocessing to the Nth drawing type content parameter set).

(Example of processing in the learning model generation unit 113)
In the first embodiment, the learning model generation unit 113 acquires the pruned, preprocessed, first drawing type content parameter set from the determination unit 115. Similar to the reference embodiment, the learning model generation unit 113 generates a plurality of learning models corresponding to the pruned, preprocessed, first drawing type content parameter set based on the pruned, preprocessed, first drawing type content parameter set. Next, the learning model generation unit 113 evaluates the quality of each of the plurality of learning models.

Specifically, the learning model generation unit 113 divides each data set included in the post-pruning and post-preprocessing first drawing type content parameter set into training data and validation data. Then, the learning model generation unit 113 uses the training data to generate multiple learning models corresponding to the training data. Next, the learning model generation unit 113 uses the validation data to evaluate the quality of each of the multiple learning models. More specifically, the learning model generation unit 113 uses the validation data to derive index values for each of the multiple learning models.

Similarly, the learning model generation unit 113 obtains the post-pruning, post-preprocessing, second drawing type content parameter set from the determination unit 115. The learning model generation unit 113 generates a plurality of learning models corresponding to the post-pruning, post-preprocessing, second drawing type content parameter set based on the post-pruning, post-preprocessing, second drawing type content parameter set. Next, the learning model generation unit 113 evaluates the quality of each of the plurality of learning models. Furthermore, the learning model generation unit 113 obtains the post-pruning, post-preprocessing, Nth drawing type content parameter set from the determination unit 115. The learning model generation unit 113 generates a plurality of learning models corresponding to the post-pruning, post-preprocessing, Nth drawing type content parameter set based on the post-pruning, post-preprocessing, Nth drawing type content parameter set. Next, the learning model generation unit 113 evaluates the quality of each of the plurality of learning models.

In embodiment 1, the learning model generation unit 113 selects the best learning model from among multiple generated learning models based on index values derived based on the pruned, post-preprocessing first drawing type content parameter set through the pruned, post-preprocessing Nth drawing type content parameter set. For example, the learning model generation unit 113 selects the learning model with the largest index value from among the multiple learning models as the best learning model. Next, the learning model generation unit 113 selects the preprocessing method corresponding to the best learning model as the best preprocessing method, similar to the reference embodiment. Subsequent processing in embodiment 1 is similar to the reference embodiment.

(Effects of the First Embodiment)
The learning device 11 according to the first embodiment can generate multiple learning models after excluding preprocessing parameters to be deleted (preprocessing parameters that are feared to have a high risk of causing multicollinearity) from multiple preprocessing parameters, and then can find the best learning model from among the multiple learning models.

In this specification, the data set obtained by deleting the preprocessed content parameters to be deleted in a preprocessed content parameter set (more specifically, the preprocessed drawing content parameter set to be searched) from that content parameter set is referred to as the pruned preprocessed content parameter set (more specifically, the pruned preprocessed drawing content parameter set to be searched). The pruned preprocessed first drawing type content parameter set through the pruned preprocessed Nth drawing type content parameter set described above are all examples of pruned preprocessed content parameter sets.

As described above, the learning device 11 can generate multiple learning models based on multiple pruned and preprocessed content parameter sets. It can then find the best learning model from among the multiple learning models. Therefore, the learning device 11 can obtain a learning model from which the effects of multicollinearity have been eliminated as the best learning model. Therefore, it is possible to obtain a best learning model of even higher quality than the reference form. In this way, the learning device 11 can improve the quality of learning models for drawing search compared to conventional methods.

Furthermore, the learning device 11 can generate multiple learning models so that only the residual preprocessed content parameters are used as explanatory variables in the learning phase. In other words, multiple learning models can be generated after reducing the number of dimensions of each dataset compared to the reference form. Therefore, the computational cost required for the learning phase can also be reduced compared to the reference form. For example, a high-quality best learning model can be obtained in a shorter computation time than the reference form.

Furthermore, the learning device 11 can select the preprocessing method corresponding to the best learning model, from which the effects of multicollinearity have been eliminated, as the best preprocessing method. Therefore, it is expected that a more effective preprocessing method for improving the accuracy of drawing search can be selected as the best preprocessing method.

The drawing search device 12 in embodiment 1 can perform drawing searches using the best learning model and best preprocessing method selected by the learning device 11 as described above. As a result, drawing searches can be performed with even higher search accuracy than in the reference embodiment.

[Modification]
In the first embodiment, the coefficient of determination (Rij) is used as the multicollinearity evaluation value. However, as will be apparent to those skilled in the art, the multicollinearity evaluation value is not limited to the above example. For example, the learning device 11 may use the above-mentioned absolute value of the correlation coefficient as the multicollinearity evaluation value.

In this case, the determining unit 115 may calculate the absolute value of the correlation coefficient between any two different preprocessed content parameters included in the data set of interest, similarly to the first embodiment. In this modification, the determining unit 115 calculates _two absolute values of the correlation coefficient _P. In this way, the determining unit 115 calculates the absolute value of the correlation coefficient (i.e., |sij|) for each preprocessed content parameter.

The determination unit 115 then compares each calculated correlation coefficient absolute value with a predetermined threshold sth (correlation coefficient absolute value threshold). The correlation coefficient absolute value threshold is another example of a multicollinearity threshold. In this specification, correlation coefficient absolute values that satisfy the condition that "the correlation coefficient absolute value is equal to or greater than sth" are referred to as high-risk correlation coefficient absolute values.

In this modified example, the determination unit 115 compares each correlation coefficient absolute value with sth for each preprocessed content parameter, and extracts high-risk correlation coefficient absolute values from among the correlation coefficient absolute values. Then, the determination unit 115 counts the number of high-risk correlation coefficient absolute values extracted for each preprocessed content parameter.

As is clear from the correspondence with the above formula (5), for example,
sth=Rth ^1/2 ...(6)
Therefore, sth is preferably set to a predetermined value equal to or greater than 0.7 and equal to or less than 1. As an example, sth may be set to 0.7. However, as will be apparent to those skilled in the art, sth is not limited to the above example.

In this modified example, the determination unit 115 generates a table (a correlation coefficient absolute value/high risk correlation coefficient absolute value count table) that indicates the correspondence between each correlation coefficient absolute value and the number of high risk correlation coefficient absolute values. This correlation coefficient absolute value/high risk correlation coefficient absolute value count table corresponds to TB6 described above. Then, as in embodiment 1, the determination unit 115 repeatedly updates the correlation coefficient absolute value/high risk correlation coefficient absolute value count table until the number of all high risk correlation coefficient absolute values in the correlation coefficient absolute value/high risk correlation coefficient absolute value count table becomes 0. Subsequent processing is the same as in embodiment 1.

As is clear from the above explanation, in one aspect of the present invention, the determination unit 115 calculates a multicollinearity assessment value for each preprocessed content parameter. Then, the determination unit 115 compares each calculated multicollinearity assessment value with a predetermined multicollinearity threshold. Specifically, the determination unit 115 compares each multicollinearity assessment value with the multicollinearity threshold and extracts high-risk multicollinearity assessment values from among the multicollinearity assessment values. The determination unit 115 counts the number of high-risk multicollinearity assessment values extracted for each preprocessed content parameter.

Next, the determination unit 115 generates a table (multicollinearity evaluation value/high-risk multicollinearity evaluation value count table) that shows the correspondence between each multicollinearity evaluation value and the number of high-risk multicollinearity evaluation values. This multicollinearity evaluation value/high-risk multicollinearity evaluation value count table corresponds to TB6 described above. Then, as described above, the determination unit 115 repeatedly updates the multicollinearity evaluation value/high-risk multicollinearity evaluation value count table until the counts of all high-risk multicollinearity evaluation values in the multicollinearity evaluation value/high-risk multicollinearity evaluation value count table become zero.

As described above, the determination unit 115 may be configured to determine preprocessed content parameters to be deleted from among a plurality of preprocessed content parameters based on a plurality of multicollinearity evaluation values derived for the plurality of preprocessed content parameters. This enables the determination unit 115 to delete preprocessed content parameters to be deleted from the plurality of preprocessed content parameters. As a result, as described above, a learning model from which the effects of multicollinearity have been eliminated can be obtained as the best learning model.

As will be clear to those skilled in the art, the method for selecting the preprocessed content parameters to be deleted is not limited to the example in embodiment 1. If the multicollinearity evaluation value between two different preprocessed content parameters is equal to or greater than the multicollinearity threshold, the determination unit 115 may determine one of the two search target drawing content parameters as the preprocessed content parameter to be deleted.

[Embodiment 2]
22 is a block diagram showing the configuration of the main parts of an information processing system 100V according to the second embodiment. The information processing device of the information processing system 100V is referred to as an information processing device 1V. The control device of the information processing device 1V is referred to as a control device 10V. The learning device and drawing search device of the control device 10 are referred to as a learning device 11V (model generation device) and a drawing search device 12V, respectively.

Unlike the learning device 11 of embodiment 1, the learning device 11V does not have a learning preprocessing unit 114. The determination unit and learning model generation unit of the learning device 11V are referred to as the determination unit 115V and the learning model generation unit 113V (learning unit), respectively. Furthermore, unlike the drawing search device 12 of embodiment 1, the drawing search device 12V does not have a search preprocessing unit 125. The search unit of the drawing search device 12V is referred to as the search unit 126V.

As is clear from FIG. 22, unlike the learning device 11, the learning device 11V does not expand each content parameter set through preprocessing. For this reason, in the second embodiment, the risk of multicollinearity occurring between multiple explanatory variables (between multiple content parameters in the example of the second embodiment) is expected to be lower than in the first embodiment. However, depending on the content of each past drawing, for example, it is possible that multicollinearity may occur between multiple content parameters. Therefore, in the second embodiment, a determination unit 115V is provided to eliminate the effects of multicollinearity.

(Example of processing in learning device 11V)
In the second embodiment, the determination unit 115V acquires the first to Nth drawing type content parameter sets from the past drawing content parameter acquisition unit 112. The determination unit 115V may calculate a multicollinearity evaluation value between any two different content parameters included in the first drawing type content parameter set.

In the second embodiment, the determination unit 115V calculates a multicollinearity evaluation value for each content parameter included in the first drawing type content parameter set (the first to Lth content parameters in the first drawing type content parameter set). That is, in the second embodiment, the determination unit 115V calculates _L C _two multicollinearity evaluation values for the first drawing type content parameter set. In this way, the determination unit 115 calculates a multicollinearity evaluation value for each combination pattern of two different drawing content parameters to be searched included in the first drawing type content parameter set.

The determination unit 115V then compares each calculated multicollinearity evaluation value with a multicollinearity threshold and extracts high-risk multicollinearity evaluation values from among the multicollinearity evaluation values. The determination unit 115 then counts the number of extracted high-risk multicollinearity evaluation values for each content parameter.

The determination unit 115V then generates a table of multicollinearity assessment values and the number of high-risk multicollinearity assessment values. The determination unit 115V then repeatedly updates the table of multicollinearity assessment values and the number of high-risk multicollinearity assessment values until the number of all high-risk multicollinearity assessment values in the table of multicollinearity assessment values and the number of high-risk multicollinearity assessment values becomes 0.

In this way, the determination unit 115V repeatedly updates the multicollinearity evaluation value/high-risk multicollinearity evaluation value count table to identify content parameters to be deleted from among the first through Lth content parameters in the first drawing type content parameter set. Content parameters to be deleted can be said to be content parameters that are feared to pose a high risk of causing multicollinearity. Note that content parameters to be deleted may also be referred to as content parameters not to be learned.

As is clear from the above explanations, the determination unit 115V may determine one of the two search target drawing content parameters as a content parameter to be deleted if the multicollinearity evaluation value is equal to or greater than a predetermined multicollinearity threshold. The determination unit 115V then generates a pruned first drawing type content parameter set by deleting the content parameter to be deleted from the first through Lth content parameters in the first drawing type content parameter set.

As an example, consider the case where, of the first through Lth content parameters in the first drawing type content parameter set, the second content parameter "current value" is identified as the content parameter to be deleted. In this case, the determination unit 115V generates a pruned first drawing type content parameter set by deleting the series corresponding to the second content parameter "current value" (the second row in the example of Figure 3) from the first drawing type content parameter set.

Similarly, the determination unit 115V calculates a multicollinearity evaluation value for each content parameter included in the second drawing type content parameter set (the first to Lth content parameters in the second drawing type content parameter set).The determination unit 115 then prunes the second drawing type content parameter set based on each multicollinearity evaluation value, thereby generating a pruned second drawing type content parameter set.

The determination unit 115V also calculates a multicollinearity evaluation value for each content parameter included in the Nth drawing type content parameter set (the 1st to Lth content parameters in the Nth drawing type content parameter set). The determination unit 115V then generates a pruned Nth drawing type content parameter set by pruning the Nth drawing type content parameter set based on each multicollinearity evaluation value. The determination unit 115V supplies the pruned 1st to Nth drawing type content parameter sets generated as described above to the learning model generation unit 113V.

In embodiment 2, the learning model generation unit 113V generates a learning model based on the pruned first drawing type content parameter set through the pruned Nth drawing type content parameter set. Specifically, the learning model generation unit 113V executes a predetermined machine learning algorithm to generate a learning model based on the pruned first drawing type content parameter set through the pruned Nth drawing type content parameter set. The learning model generation unit 113V supplies the generated learning model to the drawing search device 12V (more specifically, the search unit 126V).

The machine learning algorithm in embodiment 2 may be any one of the multiple types of machine learning algorithms exemplified in embodiment 1. Therefore, as an example, the learning model generation unit 113V may generate a learning model based on the pruned first drawing type content parameter set through the pruned Nth drawing type content parameter set using multinomial logistic regression.

As described above, in embodiment 2, each content parameter set is not expanded by preprocessing. Therefore, in embodiment 2, the best learning model and best preprocessing method are not determined. As is clear from this, in one aspect of the present invention, the steps of generating multiple learning models and selecting the best learning model from among the multiple learning models are not required. Similarly, in one aspect of the present invention, the step of determining the best preprocessing method from among multiple preprocessing methods is not required.

(Example of processing in the drawing search device 12V)
The search unit 126V (i) acquires a new drawing content parameter set from the new drawing content parameter acquisition unit 122, and (ii) acquires the learning model generated by the learning model generation unit 113V from the learning device 11V.

The search unit 126V inputs the new drawing content parameter set into the learning model. Then, the search unit 126V obtains the output of the learning model corresponding to the new drawing content parameter set from the learning model.

As an example, the search unit 126V inputs a new drawing content parameter set into a learning model, causing the learning model to output a relevance score corresponding to the new drawing content parameter set. Then, the search unit 126V searches for at least one past drawing corresponding to the drawing ND based on the output of the learning model (e.g., the relevance score).

(Effects of the Second Embodiment)
The learning device 11V in the second embodiment can generate a learning model after excluding content parameters to be deleted (content parameters that are feared to have a high risk of causing multicollinearity) from a plurality of content parameters. That is, the learning device 11V can generate a learning model based on a plurality of content parameters excluding the content parameters to be deleted.

In this specification, the data set obtained by deleting the deletion target content parameters from the content parameter set (more specifically, the search target drawing content parameter set) is also referred to as the pruned content parameter set (more specifically, the pruned search target drawing content parameter set). As described above, the learning device 11V can generate a learning model based on the pruned content parameter set. Therefore, the learning device 11V can obtain a learning model in which the effects of multicollinearity are eliminated. This makes it possible to improve the quality of the learning model compared to conventional techniques (e.g., the technology of Patent Document 1).

The drawing search device 12V in embodiment 2 can perform drawing searches using the learning model generated by the learning device 11V as described above. As a result, drawing searches can be performed with even higher search accuracy than before.

As described above, in embodiment 2, unlike embodiment 1, the selection of the best learning model and the best preprocessing method is not performed. Therefore, according to embodiment 2, the final learning model (operational model) used when operating the drawing search device can be obtained in a shorter time than in embodiment 1.

However, as is clear from the above explanation, according to embodiment 1, the best learning model can be selected as the operational model. That is, according to embodiment 1, an operational model of even higher quality can be obtained compared to embodiment 2. In addition, according to embodiment 1, the best preprocessing method can be selected as the preprocessing method to be used when the drawing search device is in operation (operational preprocessing method). By applying the best preprocessing method during operation, the drawing search accuracy of the drawing search device can be further improved. Therefore, whether the configuration of the information processing device of embodiment 1 or 2 is to be adopted may be decided appropriately by the designer of the information processing device, for example, depending on the specifications required for the information processing device.

[Software implementation example]
The functions of the information processing systems 100s, 100, and 100V (hereinafter referred to as "devices" for convenience) can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (in particular, each part included in the control device 10s, 10, and 10V).

In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in each of the above embodiments are realized by executing the program using this control device and storage device.

The above program may be stored non-transitory on one or more computer-readable storage media. These storage media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

Furthermore, some or all of the functions of each of the above control blocks can be realized by logic circuits. For example, an integrated circuit incorporating logic circuits that function as each of the above control blocks also falls within the scope of one aspect of the present invention. In addition, the functions of each of the above control blocks can also be realized by, for example, a quantum computer.

As is clear from the above explanations, the processes described in the above embodiments can be executed by AI (Artificial Intelligence). In this case, the AI may run on the control device, or on another device (such as an edge computer or cloud server).

[Additional Notes]
One aspect of the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of one aspect of the present invention.

1, 1V Information processing device 10, 10V Control device 11, 11V Learning device (model generation device)
100, 100V Information processing system 111 Past drawing data acquisition unit 112 Past drawing content parameter acquisition unit (acquisition unit)
113, 113V Learning model generation unit (learning unit)
114 Learning preprocessing unit (preprocessing unit)
115, 115V decision section

Claims (8)

A model generation device that generates a learning model for searching at least one drawing corresponding to a target drawing from among a plurality of search target drawings,
an acquisition unit that analyzes the plurality of search target drawings to acquire a content parameter set including a plurality of content parameters related to the description content of each of the plurality of search target drawings;
(i) a determination unit that calculates a multicollinearity evaluation value between two different content parameters for each combination pattern of the plurality of content parameters included in the content parameter set, and (ii) determines content parameters to be deleted from the plurality of content parameters based on the multicollinearity evaluation value;
a learning unit that generates the learning model based on a pruned content parameter set obtained by deleting the deletion target content parameters from the content parameter set ,
The model generating device
a preprocessing unit that generates a plurality of preprocessed content parameter sets each including a plurality of preprocessed content parameters by preprocessing each of the plurality of content parameters included in the content parameter set according to a combination of a predetermined plurality of types of preprocessing techniques;
The determination unit (i) calculates a multicollinearity evaluation value between two different pre-processed content parameters for each combination pattern of the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets, and (ii) determines, based on the multicollinearity evaluation value, a post-processed content parameter to be deleted from the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets;
The learning section is
generating a plurality of learning models using a plurality of pruned post-preprocessing content parameter sets obtained by deleting the post-preprocessing content parameters to be deleted from each of the plurality of post-preprocessing content parameter sets by applying each of a plurality of predetermined types of machine learning algorithms;
selecting a best learning model from the plurality of learning models based on a plurality of index values indicating the quality of each of the plurality of learning models, the index values being obtained by verifying each of the plurality of learning models using each of the plurality of post-pruning and post-preprocessing content parameter sets;
selecting a preprocessing method corresponding to the best learning model as a best preprocessing method from among the plurality of preprocessing methods;
One of the multiple preprocessing techniques is one-hot encoding.
Model generation device. 2. The model generating device according to claim 1 , wherein the learning unit selects, from the plurality of learning models, a learning model having the highest index value as the best learning model. 3. The model generating device according to claim 1, wherein the determination unit determines one of the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets as the preprocessed content parameter to be deleted when the multicollinearity evaluation value is equal to or greater than a multicollinearity threshold. The determination unit:
calculating a coefficient of determination between the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets as the multicollinearity evaluation value;
4. The model generating device according to claim 3, wherein, when the coefficient of determination is equal to or greater than a coefficient of determination threshold as the multicollinearity threshold, one of the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets is determined to be the preprocessed content parameter to be deleted. The model generating device according to claim 4 , wherein the coefficient of determination threshold is set to a predetermined value not less than 0.49 and not more than 1. The determination unit:
calculating, as the multicollinearity evaluation value, an absolute value of a correlation coefficient between the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets;
4. The model generating device according to claim 3, wherein, when the absolute value of the correlation coefficient is equal to or greater than a correlation coefficient absolute value threshold as the multicollinearity threshold , one of the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets is determined to be the preprocessed content parameter to be deleted. 7. The model generating device according to claim 6 , wherein the correlation coefficient absolute value threshold is set to a predetermined value not less than 0.7 and not more than 1. A model generation method in which a model generation device generates a learning model for searching at least one drawing corresponding to a target drawing from among a plurality of search target drawings, the method comprising:
The above model generation method is as follows:
an acquisition step in which the model generation device analyzes the plurality of search target drawings to acquire a content parameter set including a plurality of content parameters related to the description content of each of the plurality of search target drawings;
the model generating device performs a determination step of: (i) calculating a multicollinearity evaluation value between two different content parameters for each combination pattern of the plurality of content parameters included in the content parameter set; and (ii) determining a deletion target content parameter to be deleted from the plurality of content parameters based on the multicollinearity evaluation value;
a learning step in which the model generation device generates the learning model based on a pruned content parameter set obtained by deleting the deletion target content parameters from the content parameter set ,
The above model generation method is as follows:
a preprocessing step of generating a plurality of preprocessed content parameter sets each including a plurality of preprocessed content parameters by preprocessing each of the plurality of content parameters included in the content parameter set according to a combination of a plurality of predetermined types of preprocessing techniques;
the determining step further includes a step in which the model generating device (i) calculates a multicollinearity evaluation value between two different pre-processed content parameters for each combination pattern of the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets, and (ii) determines, based on the multicollinearity evaluation value, a pre-processed content parameter to be deleted from the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets;
The learning process is as follows:
The model generating device generates a plurality of learning models using a plurality of pruned post-preprocessing content parameter sets obtained by deleting the post-preprocessing content parameters to be deleted from each of the plurality of post-preprocessing content parameter sets by applying each of a plurality of predetermined types of machine learning algorithms;
a step in which the model generation device selects a best learning model from among the plurality of learning models based on a plurality of index values indicating the quality of each of the plurality of learning models, the index values being obtained by verifying each of the plurality of learning models using each of the plurality of post-pruning and post-preprocessing content parameter sets;
the model generation device further includes a step of selecting, from the plurality of types of preprocessing methods, a preprocessing method corresponding to the best learning model as a best preprocessing method;
One of the multiple preprocessing techniques is one-hot encoding.
Model generation method.