JP7663181B2 - Code generation method, code generation device, program, and data collation method - Google Patents

Description

The present invention relates to a code generation method, a code generation device, a program, and a data matching method.

Technology is known that identifies the contents of target data by comparing and matching with pre-prepared reference data. A representative example of this technology is image recognition technology (see, for example, JP 2019-28985 A). However, conventional technology requires a huge amount of learning time to obtain reference data, and the processing load required for comparing and matching with the target data is large, making high-speed processing difficult, leaving room for improvement.

JP 2019-28985 A

One of the objectives of a specific embodiment of the present invention is to provide a new technology that encodes target data such as images and audio using a small amount of data.

[1] A data matching method according to one aspect of the present invention is a method for generating and matching a code corresponding to the contents of target data using an information processing device, the method including: (a) a first step of acquiring a reference code determined according to reference data; (b) a second step of generating a specific target code corresponding to the specific target data; and (c) a third step of comparing the reference code with the specific target code to determine whether they match or do not match, and (d) the reference code in the first step includes: (d1) a step of dividing the reference data into a plurality of first sampling ranges; (d2) a step of calculating a first average value of the first element data for at least one type of the first element data among one or more types of the first element data, each of which is represented by a numerical value, that are included in each of the plurality of first sampling ranges; (d3) a step of calculating a first relative difference for each of the first sampling ranges, the first average value being divided by the sum of the first average values for each of the first sampling ranges; and (d4 ) a step of calculating a first relative difference for each of the first sampling ranges. (e) the identification code in the second step is obtained in advance through a step of generating the reference code corresponding to the reference data by concatenating, as character string data, the first relative difference for each of the first sampling ranges or a predetermined number of digits from the first relative difference, (e1) a step of dividing the identification data into a plurality of second sampling ranges set in the same manner as each of the first sampling ranges, (e2) a step of obtaining a second average value of the second element data for each of the second sampling ranges, for at least one of the second element data of the same type as the first element data, among one or more types of the second element data each of which is expressed by a numerical value, which is included in a plurality of second sampling ranges, (e3) a step of obtaining a second relative difference for each of the second sampling ranges, which is obtained by dividing the second average value of each of the second sampling ranges by the sum of the second average values of each of the second sampling ranges , ) generating the reference code corresponding to the reference data by concatenating, as character string data, the second relative difference for each of the second sampling ranges or a predetermined number of digits from the upper digit thereof, and (f) the third step includes: (f1) determining, between each of the corresponding first sampling ranges and each of the second sampling ranges, a difference between the first relative difference for each of the first sampling ranges obtained based on the reference code or a predetermined number of digits from the upper digit thereof and the second relative difference for each of the second sampling ranges obtained based on the specific target code; and (f2) determining that the reference code and the specific target code match if all of the differences fall within a predetermined allowable range.

The above configuration provides a new technology that encodes target data such as images and audio using a small amount of data.

FIG. 1 is a diagram for explaining the encoding method of the first embodiment. FIG. 2 is a diagram showing an example of the configuration of a database of codes generated by the above-mentioned method and contents corresponding to each code. FIG. 3 is a diagram conceptually explaining the flow of matching using a database. FIG. 4 is a diagram for explaining a specific method for performing flexible matching. FIG. 5 is a line graph showing the differences corresponding to each of the sampling ranges S1 to S9 in the example shown in FIG. FIG. 6 is a block diagram showing an example of the configuration of a code generating device for generating code using the above-mentioned method. FIG. 7 is a block diagram showing an example of the configuration of a user device that performs matching of target data using a code generated by the above method. FIG. 8 is a diagram showing an example of the configuration of a computer that realizes the information processing unit in each of the code generation device and the user device. 9A is a flowchart showing the operation procedure of the code generating device, and FIG 9B is a flowchart showing the operation procedure of the user device. FIG. 10 is a diagram for explaining application example 1 of the encoding technique according to the embodiment described above. FIG. 11 is a diagram for explaining application example 2 of the encoding technique according to the above embodiment. FIG. 12 is a diagram for explaining application example 3 of the encoding technique according to the embodiment described above. FIG. 13 is a diagram for explaining application example 4 of the encoding technique according to the above embodiment. FIG. 14 is a diagram for explaining application example 5 of the encoding technique according to the above embodiment. FIG. 15 is a diagram for explaining an application example 6 of the encoding technique according to the above embodiment. FIG. 16 is a diagram for explaining application example 7 of the encoding technique according to the above embodiment. FIG. 17 is a diagram for explaining the encoding method according to the second embodiment. FIG. 18 is a diagram showing an example of a code expressed in hexadecimal character code. FIG. 19 is a diagram for explaining a matching method using a code according to the second embodiment. FIG. 20 is a diagram for explaining the encoding method according to the third embodiment. FIG. 21 is a diagram for explaining a matching method using a code according to the third embodiment. FIG. 22 is a conceptual diagram showing a method of using each embodiment.

First Embodiment
1 is a diagram for explaining the coding method of the first embodiment. Here, image data is taken as an example of data to be coded (target data). In addition, coding in this specification refers to generating character string data that can identify the target data based on a predetermined numerical value included in the target data.

Assuming that the target data is image data as shown in Figure 1 (A). The image data may be a color image or a monochrome image. The image data includes values (element data) such as brightness, hue, saturation, and color difference for each pixel. For ease of understanding, the following explanation uses brightness values as element data.

As shown in Figure 1 (B), the image data is divided into multiple sampling ranges. Here, the image data is divided into nine sampling ranges S1, S2, S3, ... S9. Each sampling range S1, etc. has a width and height of, for example, 40 pixels, and the total number of pixels contained in each sampling range S1, etc. is 1,600. The brightness value (number of gradations) of each pixel is given as a number in the range from 0 to 255.

In this case, if the pixel values (here, brightness values) of the pixels included in each sampling range S1 etc. are px1, px2, . . . px1600, the average pixel values in each sampling range S1 etc. can be calculated as follows.
Sum of pixel values: sumSx = px1 + px2 + px3 + ... px1600
Average pixel value: Ax = sumSx/1600

Figure 1 (C) shows an example of the average pixel values in each sampling range S1, etc. In the example shown, the average value A1 of sampling range S1 is 128, the average value A2 of sampling range S2 is 86, the average value A3 of sampling range S3 is 134, the average value A4 of sampling range S4 is 101, the average value A5 of sampling range S5 is 68, the average value A6 of sampling range S1 is 101, the average value A7 of sampling range S7 is 115, the average value A8 of sampling range S8 is 91, and the average value A9 of sampling range S9 is 121.

Next, the relative difference in the average values in each sampling range S1 etc. is calculated as follows.
Sum of the average values: sumA = A1 + A2 + A3 + ... + A9
= 945
Relative difference of the average values: Vx = (100 x Ax) / 945

Figure 1 (D) shows an example of the relative difference of the average values in each sampling range S1, etc. In the example shown, the relative difference V1 in sampling range S1 is 13.54%, the relative difference V2 in sampling range S2 is 9.10%, the relative difference V3 in sampling range S3 is 14.17%, the relative difference V4 in sampling range S4 is 10.68%, the relative difference V5 in sampling range S5 is 7.19%, the relative difference V6 in sampling range S6 is 10.68%, the relative difference V7 in sampling range S7 is 12.16%, the relative difference V8 in sampling range S8 is 9.62%, and the relative difference V9 in sampling range S9 is 12.80%.

The relative difference in each of these sampling ranges S1, etc. is used to extract the numerical value from the most significant digit to the second decimal place of each relative difference (four digits in total) as character string data. For example, the character string data "1354" is extracted from sampling range S1. Note that if there is no value in the tens digit or in the decimal place, the value is set to 0. For example, 9.1% becomes "0910".

In this way, the character string data extracted from each sampling range S1 etc. is concatenated. There is no particular restriction on the concatenation order, and it can be determined arbitrarily. Here, the sampling ranges are concatenated in the order S1, S2, ... S9. As a result, in the above example, the character string data "135409101417106807191068121609621280" is obtained (see FIG. 1 (E)). In this embodiment, the character string data generated in this way is used as a code corresponding to the target data.

The relative difference does not necessarily have to be expressed as a percentage. The order in which the character string data corresponding to each sampling range is concatenated can be any order, provided that a predetermined order is always used. For example, it can be decided that sampling ranges are concatenated in the order S9, S8, ... S1. It is also arbitrary to what extent the relative difference is used in each sampling range. The number of divisions of the sampling range can also be arbitrary. Reducing the number of divisions of the sampling range enables simpler and faster matching, while increasing the number of divisions of the sampling range enables more accurate matching.

Figure 2 is a diagram showing an example of a database configuration of codes generated by the above method and contents corresponding to each code. Here, images are used as an example of contents. As shown in the figure, codes corresponding to each image 1, 2, etc. are generated in advance by the above method, and a database linking them is constructed. For example, image 1 corresponds to code "2110...", and image 2 corresponds to code "1053...". Here, images 1, 2, etc. as contents may be image data itself, or may include additional information corresponding to the contents of the image data (for example, a character string representing an object contained in the image, position information in real space, time information, and other arbitrary additional information). By constructing such a database in advance, it is possible to use each code in this database as reference data to check which content the specific target data corresponds to.

Figure 3 is a diagram conceptually explaining the flow of matching using a database. For example, image data obtained by photographing is coded using the above method, and a specific target code ("8841..." in the illustrated example) that is a code corresponding to this image data is generated. This generated specific target code is compared with each reference code in the database, and if a reference code that matches the generated specific target code exists in the database, the content corresponding to the photographed image ("Image 37" in the illustrated example) can be identified.

For example, assume that image data is obtained by photographing an arbitrary space using a terminal device (a smartphone, as an example) equipped with a camera. Also assume that an application program capable of executing the above-mentioned coding is pre-installed on the terminal device, and that a database such as that shown in FIG. 2 is stored in the terminal device. Note that the database may be placed on a server capable of communicating with the terminal device via a network. In this case, for example, if content corresponding to an image of a landscape photographed by a user with a camera is identified, it is possible to realize a usage such as displaying information related to that content on the screen of the user's terminal device.

In addition, according to the research of the present inventors, if the target data is image data obtained by photographing a space, it has been found that by encoding the data with a sampling range of 64 or more, it is possible to perform matching and judgment with sufficient accuracy for practical purposes. The data volume of the code generated under these conditions is about 192 bytes, which can be said to be a very small amount of data.

According to the matching principle explained using Figure 3, if all the character string data of the codes match, the content corresponding to each code is determined to be the same. However, by setting the tolerance range for the judgment as follows, more flexible matching can be performed.

Figure 4 is a diagram for explaining a specific method for performing flexible matching. Also, Figure 5 is a line graph showing the differences corresponding to each of the sampling ranges S1 to S9 in the example shown in Figure 4 above.

In the example shown in FIG. 4A, the character string data of each of the reference code A and the specific target code B is divided into four digits corresponding to the above-mentioned sampling range, each character string data is treated as a four-digit numerical value, and the numerical difference (A-B) is calculated for each corresponding sampling range. For example, the numerical value corresponding to the sampling range S1 is "1354" in the reference code and "1368" in the specific target code, so the difference between the two is "-14". Similarly, the differences between the other sampling ranges S2 to S9 are +15, 0, -12, +19, +17, +15, 0, and +18. In this case, if the allowable range of the difference corresponding to each sampling range is previously set to, for example, ±20, the differences corresponding to all of the sampling ranges S1 to S9 in the example shown in FIG. 4A fall within this allowable range. In such a case, it can be determined that the reference code and the specific target code are "identical (matching)". As shown in FIG. 5, when it can be determined that the reference code and the specific target code are the same (A and B), the graphs almost overlap.
On the other hand, in the example shown in Fig. 4(B), the differences for each sampling range S1 to S9 for the character string data of the reference code A and the specific target code C are +74, -101, +27, 0, -261, +38, +126, -249, and +273. If the allowable range of the difference corresponding to each sampling range is previously set to ±20, in the example shown in Fig. 4(B), only the difference corresponding to the sampling range S4 falls within the allowable range, and the differences corresponding to the other sampling ranges do not fall within the allowable range. In such a case, it can be determined that the reference code and the specific target code are "different (mismatched)". In this case, as shown in Fig. 5, the graphs do not overlap when it can be determined that the reference code and the specific target code are different (A and C).

If no tolerance range is set, it means that the image capture conditions (reflected light, ambient light, etc.) are exactly the same. In contrast, if a tolerance range is set, it becomes possible to derive a "solution" that corresponds to slight environmental changes such as lighting conditions and the user's senses. By using this tolerance range (environmental variable), it is possible to realize a judgment (maybe) that is close to the sense of the human eye. The size of the tolerance range can be set arbitrarily, but according to the inventor's investigation, it is preferable to set it to about ±20 as mentioned above.

Figure 6 is a block diagram showing an example of the configuration of a code generation device for generating code using the above-mentioned method. The illustrated code generation device 1 includes an information processing unit 10, a camera 11, a microphone 12, a storage device 13, an operation device 14, and a display device 15.

The information processing unit 10 performs information processing related to code generation. The information processing unit 10 includes, as functional blocks, a sampling processing unit 20, a feature value calculation unit 21, an encoding processing unit 23, and a code recording processing unit 24. The information processing unit 10 is realized by using, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and executing a predetermined operating program on the computer. In this embodiment, the sampling processing unit 20 corresponds to the "division unit", the feature value calculation unit corresponds to the "average value calculation unit" and the "relative difference calculation unit", and the encoding processing unit 23 corresponds to the "code generation unit".

The camera 11 captures an image of a space, object, etc. that may be the subject of code generation, and generates an image of the captured image. The image referred to here is not necessarily limited to an image in the visible light range, but may be, for example, a thermal image or an infrared image. The camera 11 outputs image data (or image signal) of the captured image to the information processing unit 10.

The microphone 12 picks up sounds that can be used to generate codes and converts them into electrical signals. The sounds referred to here are not limited to sounds in the audible range, but may also be ultrasonic or low-frequency sounds. The sound signals output from the microphone 12 are output to the information processing unit 10, where they are converted into digital data and imported.

The storage device 13 stores the configuration of the information processing unit 10, programs and various data required for its operation, and also stores a database of codes generated by the information processing unit 10. The storage device 13 is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

The operation device 14 is used to input information necessary for the operation of the information processing unit 10, and may include, for example, a keyboard, a mouse, a switch, a touch panel, etc. The display device 15 is used to display information related to the operation of the information processing unit 10, and may include, for example, a liquid crystal display device or an organic EL display device.

The sampling processing unit 20 performs a process of dividing the target data for code generation into multiple sampling ranges (see FIG. 1(B)). The target data is, for example, image data or sound data. In the case of sound data, the data can be divided into predetermined time intervals, and the sound data included in these intervals can be treated as data corresponding to each sampling range (see the third embodiment described below).

The feature value calculation unit 21 calculates the average value of the specified data included in each sampling range divided by the sampling processing unit 20 (see FIG. 1(C)), and calculates the relative difference of the average values corresponding to each sampling range (see FIG. 1(D)). The relative difference is expressed as a percentage, for example, as in the above example, but is not limited to this.

The encoding processing unit 23 uses the relative difference (or average value) corresponding to each sampling range calculated by the feature value calculation unit 21 to generate a code corresponding to the target data (a code formed by concatenating character string data).

The code recording processing unit 24 associates the code generated by the encoding processing unit 23 with information data that identifies the content corresponding to this code and records them in the storage device 13 to generate a code database (see Figure 2).

Figure 7 is a block diagram showing an example of the configuration of a user device that uses a code generated by the above method to match target data. The illustrated user device 2 includes an information processing unit 50, a camera 51, a microphone 52, a storage device 53, an operation device 54, and a display device 55. This user device 2 is realized by pre-installing a specific application program on an information processing device such as a smartphone owned by the user.

The information processing unit 50 performs information processing related to code matching. The information processing unit 50 has the following functional blocks: a sampling processing unit 60, a feature value calculation unit 61, an encoding processing unit 63, and a matching processing unit 64. The information processing unit 50 is realized by using a computer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and executing a predetermined operating program on the computer.

The configurations and operations of the camera 51, microphone 52, storage device 53, operation device 54, and display device 55 are similar to those of the camera 11, microphone 12, storage device 13, operation device 14, and display device 15 in the code generation device 1 described above, so detailed descriptions will be omitted here. The configurations and operations of the sampling processing unit 60, feature value calculation unit 61, and encoding processing unit 63 are similar to those of the sampling processing unit 20, feature value calculation unit 21, and encoding processing unit 23 in the code generation device 1 described above, so detailed descriptions will be omitted here.

The matching processing unit 64 of the information processing unit 50 matches the specific target code obtained by the processing of the sampling processing unit 60, the feature value calculation unit 61, and the encoding processing unit 63 with a database (see FIG. 2) pre-stored in the storage device 53, and identifies the content corresponding to the specific target code. The specific matching method used by the matching processing unit 64 may be a method that requires a perfect match between the codes, or may be a flexible method that sets a tolerance for the differences between the codes as described above. The identification result is displayed on the display device 55, for example.

Figure 8 is a diagram showing an example of the configuration of a computer that realizes the information processing unit in each of the code generation device and the user device. The computer shown in the figure is configured to include a CPU 201, a ROM 202, a RAM 203, a storage device 204, and an external interface (I/F) 205 that are connected to each other so that they can communicate with each other. The CPU 201 operates based on a basic control program read from the ROM 202, and realizes the function of the controller 13 described above by reading and executing a program (application program) 206 stored in the storage device 204. The RAM 203 temporarily stores data to be used when the CPU 201 operates. The storage device 204 is a non-volatile storage device such as a hard disk or a solid state drive, and stores various data such as the program 206. The external interface 205 is an interface that connects the CPU 201 to an external device, and is used to connect the CPU 201 to, for example, the camera 11 (51), the microphone 12 (52), the operation device 14 (54), and the display device 15 (55).

Figure 9 (A) is a flowchart showing the operation procedure of the code generation device. Figure 9 (B) is a flowchart showing the operation procedure of the user device. Note that in either operation procedure, the order of processing can be changed as appropriate as long as no contradictions or inconsistencies occur in the results of information processing. In addition, other processing not mentioned here can be added, and these implementations are not excluded.

First, the operation procedure of the code generating device will be described.
When target data is input to the code generation device 1 (step S11), the sampling processing unit 20 performs a process of dividing the target data into a plurality of sampling ranges (step S12). Here, the target data may be image data input from the camera 11, sound data corresponding to a signal input from the microphone 12, or image data or sound data previously stored in the storage device 13. The target data may also be data transmitted from a server or the like (not shown) via a network and received by the code generation device 1.

Next, the feature value calculation unit 21 calculates the average value of predetermined element data included in each sampling range divided by the sampling processing unit 20 (step S13). The element data here may be any data that is included in the target data as described above and that forms the contents, features, or characteristics of the target data. For example, as described above, in the case of image data, it may be the brightness of each pixel, and in the case of sound data, it may be the volume of sound at each predetermined sampling time.

Next, the feature value calculation unit 21 calculates the relative difference between the average values corresponding to each sampling range (step S14). The relative difference is expressed, for example, as a percentage as in the above example, but is not limited to this.

Next, the encoding processing unit 23 generates a reference code (a code formed by concatenating character string data) corresponding to the target data using the relative difference corresponding to each sampling range calculated by the feature value calculation unit 21 (step S15).

Next, the code recording processing unit 24 associates the reference code generated by the encoding processing unit 23 with information data identifying the content corresponding to this reference code and records them in the storage device 13 to generate a database of reference codes (see Figure 2) (step S16).

By repeating the above series of information processing as many times as necessary, reference codes corresponding to each of the multiple contents are generated, and a database containing that information is obtained. The generated database may be transmitted to and stored in a server device (not shown) or the like that is communicatively connected via a network. This allows the database to be shared by multiple users via the network.

Next, the operation procedure of the user equipment will be described.
When specific target data is input to the user device 2 (step S21), the sampling processing unit 60 performs processing to divide the specific target data into a plurality of sampling ranges (step S22). The specific target data here may be image data input from the camera 11, sound data corresponding to a signal input from the microphone 12, or image data or sound data previously stored in the storage device 13.

Next, the feature value calculation unit 61 calculates the average value of predetermined element data included in each sampling range divided by the sampling processing unit 60 (step S23). The element data here may also be data that is included in the target data as described above and is used to form the contents, features, or characteristics of the target data.

Next, the feature value calculation unit 61 calculates the relative difference between the average values corresponding to each sampling range (step S24).

Next, the encoding processing unit 63 generates a specific target code (a code formed by concatenating character string data) corresponding to the specific target data using the relative difference corresponding to each sampling range calculated by the feature value calculation unit 61 (step S25).

Next, the matching processing unit 64 refers to a database previously stored in the storage device 53 to identify the content corresponding to the identification target code generated in step S25 (step S26). The identification result is displayed on the display device 55 as appropriate. The database stored in the storage device 53 is, for example, transmitted from a server or the like (not shown) via a network and received by the user device 2.

The above is the basic content regarding the code generation and matching of the first embodiment, and next, some application examples will be explained. Note that the code generation and matching related to the following application examples are all performed by the above-mentioned code generation device 1 and user device 2 (or other device).

Figure 10 is a diagram for explaining application example 1 of the coding technology according to the embodiment described above. Here, an application example to face recognition technology is shown. The face recognition technology here refers to a technology for identifying a person corresponding to a face included in a certain image. The certain image may be a still image or a moving image taken by a camera. In the case of a moving image, a still image obtained for each frame can be used. As shown in Figure 10 (A), assume that face image data 300 of a certain person from the front is given. For this face image data 300, face image data corresponding to a bird's-eye view of the person from each of the up, down, left, and right directions is automatically generated using a publicly known technology. Of course, the face image data may not be automatically generated, but may be directly captured from the person. Here, a total of 15 face image data are shown, including the front face image data 300 and face image data corresponding to each of the up, down, left, and right directions that are automatically generated (or captured) around it.

These facial image data are coded by the code generation device 1 as shown in FIG. 10(B) to obtain reference codes. Here, code 1 is obtained, which corresponds to the frontal facial image data 300, and codes 2 to 15 are obtained, which correspond to the other facial image data in the up, down, left and right directions. These 15 codes 1 to 15 are linked as a set of facial recognition data. For example, each code can be given identification information such as "xxxxxx_CEC" as identification information (FaceID) and linked to these. In addition, attribute information (such as name, age, affiliation, etc.) related to the person to be identified is linked to the identification information.

By generating and storing such face recognition data in advance, face recognition can be performed in the user device 2 (or other device) in real time based on image data obtained from a camera. There are various specific matching methods, and for example, a person may be identified if any one of Code 1 to Code 15 matches, or a condition may be that multiple codes match. The code obtained by the coding technology of this embodiment is irreversible character string data (text data) and does not use face image data itself, so that privacy can be protected even if information is leaked, for example, when face recognition data is stored on a network server. In addition, since the data size of one code is several hundred bytes, the data capacity required for face recognition can be dramatically reduced. For example, if the data size of one face image data is 100 kilobytes, the data capacity of 15 face image data is 1500 kilobytes, whereas the data size of one code can be about 192 bytes, so the data capacity of 15 codes is 2880 bytes, and the difference in data capacity is obvious.

Figure 11 is a diagram for explaining an application example 2 of the coding technique according to the embodiment described above. Here, an application example is shown in which image data such as space is divided into a plurality of partial images, and a code is generated for each partial image to perform image recognition (spatial recognition). Specifically, as shown in Figure 11 (A), the code generation device 1 divides image data 400 as target data into a plurality (here, 12) of partial images A to L. Then, the above coding is performed for each of the partial images A to L. The code generated corresponding to partial image A is represented as A_code. The same applies to the codes corresponding to the other partial images. The 12 codes such as A_code generated in this way are linked and recorded as one reference code group.

Similarly, the image data as identified target data is divided into multiple partial images by the user device 2 (or another device) and a code corresponding to each partial image is generated. For example, code generation is performed for a still image captured in real time by the user's camera, or for one frame of a moving image. At this time, for example, as shown in FIG. 11(B), it is assumed that the shooting range of the image data 400 corresponding to the reference data group and the image data 410 as identified target data by the user partially overlap. Here, it is assumed that the portions corresponding to partial images E, F, I, and J overlap.

In this state, if the code group generated from the image data 410, which is the specific target data, is referenced to the reference code group in a brute force manner, the number of matching partial images T=4 and the number of mismatching partial images F=8 will be the total number of partial images N=12 (see FIG. 11C). If the consistency rate (recognition rate) is defined as T/N and the mismatch rate is defined as F/N, in this example, the consistency rate is about 33% (T/N=4/12) and the mismatch rate is about 66% (F/N=8/12). The consistency rate at which the specific target data and the reference data are judged to match varies depending on the application, and can be set appropriately. The consistency rate can also be used to judge similarity. For example, if the consistency rate is about 33% as described above, it can be judged that there is a high possibility that they are similar, or that they are photographed in a close place. Using such a method, any frame-in can be handled. Also, the greater the total number N of partial images, the more the recognition accuracy can be improved. Even in this case, the search, matching, and judgment are basically performed using text data, so the same high-speed processing as commonly known text searches can be performed.

Figure 12 is a diagram for explaining application example 3 of the coding technology according to the embodiment described above. Here, an application example is shown in which the up-down direction of an object is determined. There is no particular restriction on the content of the object as long as it has an asymmetric shape in the up-down direction. For example, there is a case where the up-down direction is determined in industrial machinery to crack pistachios.

As shown in FIG. 12(A), an object 500 is divided into two parts a and b by focusing on its asymmetry in the vertical direction. As an example, part a is regarded as a triangle, and part b as a trapezoid. The base of part a is equal to the bottom base of part b, and the heights of both are also equal. When parts a and b are fitted into regions 510 and 512 of the same size (rectangular regions indicated by dotted lines in the figure) and the remaining parts other than parts a and b are compared, the remaining part of rectangular region 510 corresponding to part a is larger than that of rectangular region 512 corresponding to part b. Based on this knowledge, coding technology is applied.

Specifically, as shown on the left side of FIG. 12(B), first, an image of a base area is captured without the presence of an object 500, and the image data 520 is divided into multiple partial images (several hundred, for example), a reference code corresponding to each partial image is generated, and the reference codes are recorded.

Next, as shown in the center of Figure 12(B), the edges of the object 500 are detected based on image data 522 obtained by photographing the base area with the object 500 placed in that area, and an effective area 524 (rectangular area indicated by a thick line in the figure) in which the object 500 exists is extracted. As shown in Figure 12(B), the inside of this extracted rectangular area 524 is divided so that the upper and lower halves each have the same number of partial images, and each partial image is coded. The partial images are divided in the same way as when the reference code group was generated.

Next, when the specific object code group generated with the object 500 placed is compared with the reference code group, the codes corresponding to the area where the object is present do not match, but the codes corresponding to the other areas match. In FIG. 12(B), the areas where the codes match are shown with a diagonal line pattern. As shown in FIG. 12(C), the number of areas where the codes match is counted separately for the upper half area 524a of the effective area and the lower half area 524b of the effective area. In the illustrated example, 24 codes match in the upper half area 524a of the effective area, and 13 codes match in the lower half area 524b of the effective area. In this case, the upper half area 524a of the effective area corresponds to the above-mentioned part a, so it can be determined that it is "upper", and the lower half area 524b of the effective area corresponds to the above-mentioned part b, so it can be determined that it is "lower".

Figure 13 is a diagram for explaining an application example 4 of the coding technique according to the embodiment described above. Here, an application example is shown in which an object (an animal, as an example) in a video (image) is coded in advance, and the object present in the video is recognized using the reference code group. Specifically, first, as shown in Figure 13 (A), edge detection is performed on an object 600 (an animal in this case) in the video using a known technique, and an edge 610 of the object is extracted as shown in Figure 13 (B). Next, as shown in Figure 13 (C), multiple rectangular areas are mapped to the extracted edge 610 and its internal area 612. The number of rectangular areas can be set arbitrarily, and the greater the number, the higher the recognition accuracy. Then, for the partial images of each mapped rectangular area, a reference code corresponding to each partial image is generated by the above-mentioned method, and the reference code group is recorded. In the same manner, a reference code group corresponding to each shooting direction of the object is generated and recorded. Information about the object (for example, attribute information such as the name of an animal) is linked to these reference code groups and recorded.

Next, for example, in video captured in real time, the system performs edge detection of objects and mapping of rectangular areas in the same manner as described above, and encodes the partial images of each rectangular area to generate a set of specific object codes corresponding to the objects in the video. By comparing this set of specific object codes with a set of reference codes recorded in advance, it is possible to recognize objects contained in the video and identify their presence or absence, their names, and other attributes.

In this way, by combining existing edge detection technology with the coding technology of this embodiment, it becomes possible to instantly register and match codes, improving recognition accuracy. The same is true when the target object is space or electronic data. The coding technology of this embodiment can achieve real-time generation, registration, comparison, matching, and judgment, improving recognition accuracy and speed.

Figure 14 is a diagram for explaining application example 5 of the coding technique according to the embodiment described above. Here, for example, an application example to an appearance inspection process during the manufacture of industrial products is shown. Specifically, first, as shown in Figure 14 (A), the appearance of the product to be inspected is photographed to obtain image data 700. Next, as shown in Figure 14 (B), the image data 700 is rotated 360 degrees at any angle (for example, 1 degree) to obtain image data 701, 702, etc. at each angle. Then, coding is performed based on the image data at each angle, and each code is recorded as a reference code. Next, as shown in Figure 14 (C), in the actual appearance inspection process, for example, codes are generated from image data 710 to 713 obtained by photographing each product moving on a belt, and these are compared with the reference code recorded in advance. As a result, even if the angle at which the product is arranged is not constant, if the shape and color tone are the same, it can be judged as an acceptable product (OK product), and if there is a partial chip or a difference in color tone, it can be judged as a defective product (NG product). In the illustrated example, the products corresponding to image data 711 and 713 are acceptable products, and the products corresponding to image data 710 and 713 are deemed defective.

When generating reference codes in advance in this way, if one good product (a product without defects) is photographed in one position (angle), multiple reference codes that can be identified even if the direction or angle changes can be automatically generated through in-memory processing. Also, by embedding color information in the code, it becomes possible to judge not only by shape but also by color tone.

Figure 15 is a diagram for explaining application example 6 of the coding technology according to the embodiment described above. For example, if it is determined that products will flow in a fixed arrangement during the visual inspection process, it is not necessary to generate reference codes with different angles. In this case, a single reference code corresponding to a non-defective product in a fixed arrangement can be used. Here, as an example, consider the case of detecting defects such as missing parts, printing problems, and scratches on a computer keyboard.

First, as shown in FIG. 15(A), the appearance of the product to be inspected is photographed to obtain image data 800, which is then coded and recorded as a reference code. At this time, as shown in FIG. 15(B), it is preferable to divide the image data 800 of the target product into a number of rectangular regions to obtain partial images 801 corresponding to each region, and obtain a reference code group, which is a set of reference codes corresponding to each partial image 801. Next, in the actual appearance inspection process, for example, a specific target code (group) is generated from the image data obtained by photographing each product moving on a belt, and this is compared with the reference code (group) recorded in advance. This makes it possible to detect defects such as missing keyboards, printing problems, and scratches, if any. In this case, as shown in FIG. 15(C), it is also possible to identify the area 810 where the defect exists based on the mismatched reference code.

Figure 16 is a diagram for explaining application example 7 of the coding technique according to the embodiment described above. Here, a method for determining the shape and area of an object is explained. The object is assumed to be placed in a fixed shooting area and photographed. In this case, image data of the shooting area is first obtained without the object being placed, and then this is divided into multiple areas, and partial images of each area are coded. This results in a reference code group consisting of multiple (81 in the illustrated example) codes Cd11, Cd12, ..., Cd99, as shown in Figure 16 (A), and this reference code group is recorded.

Next, image data of the photographed area where the object is located is obtained, divided into multiple areas, and partial images of each area are coded. This results in a specific object code group. This is compared with the reference code group. Then, as shown in FIG. 16B, an area 901 where the codes match (a rectangular area with no pattern in the figure) and areas 902 and 903 where the codes do not match (rectangular areas with patterns in the figure) are obtained. The areas 902 and 903 where the codes do not match are areas where the object exists. Here, the matching rate is divided in 10% increments. Among the areas where the codes do not match, for example, the area 902 with a matching rate of 30% to 40% in the illustrated example is connected to obtain the edge of the object, that is, the shape. In addition, the area within this edge can be obtained to obtain the area of the object in a planar view. In addition, the unevenness of the object can be estimated based on the area 903 with a different matching rate within the edge (here, for example, a matching rate of 60% to 70%). There are various ways to calculate the match rate between codes, but a simple example is to compare the codes in order, character by character, count the number of matching characters, and divide this number by the total number of characters. For example, if a code contains 384 characters and 370 of them match, the match rate is about 96%.

Second Embodiment
Fig. 17 is a diagram for explaining the coding method of the second embodiment. Here, image data is taken as an example of data to be coded (target data) for explanation. Note that the configurations of the code generation device and user device are the same as those of the first embodiment (see Figs. 6 and 7), and only the information processing method relating to code generation and matching is different. Here, explanation of the device configuration is omitted, and the information processing method for code generation and matching will be described in detail.

As shown in FIG. 17(A), image data as target data (reference data) is divided into multiple sampling ranges. In the illustrated example, it is divided into 64 sampling ranges S11, S12, S13, ... S88 of 8 rows and 8 columns, but the number of divisions is not limited to this. The total number of pixels contained in each sampling range S11 to S88 is, for example, 1000 to 2000. Each pixel contains color information in YUV format based on a known format. Here, Y represents the luminance value, U (Cb) represents the difference between the luminance value and the blue component, and V (Cr) represents the difference between the luminance value and the red component. Hereinafter, the respective values are referred to as the Y value, U value, and V value. This process is performed by the sampling processing units 20 and 60.

At this time, the average values of the Y, U and V values are calculated for each of the sampling ranges S11 to S88. The method of calculating the average values is the same as in the first embodiment described above. For example, for sampling range S11, the average values are calculated by calculating the sum of the Y, U and V values of all pixels contained in this sampling range S11 and dividing the sum by the total number of pixels in sampling range S11. The same applies to the other sampling ranges. This process is performed by feature value calculation units 21 and 61.

Figure 17 (B) shows an example of the average values of the Y, U, and V values for each sampling range. Here, the average values of the Y, U, and V values for sampling range S11 are represented as Y11, the average value of the U values as U11, and the average value of the V values as V11, in order to represent the average values of the Y, U, and V values for each sampling range. For example, in sampling range S12, the average values are Y12 = 119, U12 = 128, and V12 = 142.

A code is generated using the average values of the Y, U, and V values of each sampling range obtained as described above. As shown in FIG. 17C, for example, the average values are linked in the order of Y, U, and V values as follows: Y11, Y12, Y13, ... Y21, Y22, ... Y87, Y88, U11, U12, U13, ... U21, U22, ... U87, U88, V11, V12, V13, ... V21, V22, ... Y87, Y88. Note that this is just an example, and the order of linking the Y, U, and V values can be set as appropriate. In this way, a reference code corresponding to the image data as the target data (reference data) is generated. This process is performed by the encoding processing units 23 and 63.

If the values of Y11 to Y88, U11 to U88, and V11 to V88 are each expressed as a hexadecimal string, for example, then the data size of the code obtained when a sampling range of 64 divisions with 8 rows and 8 columns is set as above will be 192 bytes (= 64 x 3). This corresponds to 384 characters of text data. An example of a code expressed in hexadecimal character code is shown in Figure 18. In hexadecimal characters, the data amount per character is 4 bits, so although the number of characters is 384, the data amount when recorded will be 192 bytes.

According to the inventors' research, this 192-byte data volume is the practical minimum value for performing comparison, matching, judgment, and recognition in real time without complex calculations. In other words, 192 bytes is the minimum value of an effective code that can achieve a recognition rate of about 98% for visualized target data such as space, landscape, object, heat source video, video, and images. This data size does not depend on the area or data size of the original video or image.

For example, consider the amount of data when 24 codes are generated per second to correspond to a video with a frame rate of 24 fps. If the data size of a code corresponding to one image is 192 bytes as shown above, the number of codes corresponding to a 30-minute video will be 43,200, with the amount of data being approximately 7.9 MB. Similarly, the number of codes corresponding to a 1-hour video will be 86,400, with the amount of data being approximately 15.8 MB, the number of codes corresponding to a 2-hour video will be 172,800, with the amount of data being approximately 31.6 MB, and the number of codes corresponding to a 1-day video will be 2,073,600, with the amount of data being approximately 379.7 MB. Even if each code is accompanied by associated information such as coordinates (longitude, latitude, altitude: 24 bytes), time (date and hour, minute, second: 8 bytes), and a comment of 100 double-byte characters (200 bytes), the amount of data corresponding to a 30-minute video is approximately 17.5 MB, the amount of data corresponding to a 1-hour video is approximately 34.9 MB, the amount of data corresponding to a 2-hour video is approximately 69.9 MB, and the amount of data corresponding to a 1-day video is approximately 838.5 MB. Note that if one code is generated corresponding to one frame per second, the amount of data will be 1/24 of the above amount. In this case, the amount of data for a 1-day video will be very small, at 15.8 MB (without associated information) or 34.9 MB (with associated information).

In addition, since this code contains color information for each of YUV, it is possible to obtain color information in RGB format by using this color information through the following conversion formula:
R = (1.164 x (Y value - 16)) + (1.596 x (V value - 128))
G = (1.164 x (Y value - 16)) - (0.391 x (U value - 128)) - (0.813 x (V value - 128))
B = (1.164 x (Y value - 16)) + (2.018 x (U value - 128))

Figure 19 is a diagram for explaining the matching method using the code of the second embodiment. Here, to simplify the explanation, consider a case where a sampling range of 3 rows and 3 columns is set. In the figure, the left side shows a reference code that has been generated and recorded in advance, and the right side shows a specific target code obtained from the specific target data. Note that each code is actually a character string as shown in Figure 17 (C) above, but here it is expressed so that the correspondence between each sampling range is easy to understand. Possible matching methods include, for example, a simple comparison method and a relative comparison method. These processes are executed by the matching processing unit 64.

First, a simple comparison method is described. For each sampling range, the differences between the Y value, U value, and V value of the reference code and the specific target code are calculated, and the differences are squared. For example, in the illustrated example, the square of the Y value difference in the sampling range of row 1, column 1 is (202-195) ² =49. Similarly, the square of the Y value difference in the sampling range of row 1, column 2 is (195-212) ² =289. Similarly, the square of the Y value difference in the sampling range of row 3, column 3 is (149-190) ² =1681. Similarly, the square of the Y value difference is calculated for the other sampling ranges, and the sum of these is calculated. In the illustrated example, the sum of the squares of the Y value differences in each sampling range is 15114 (=49+289+...+1681). Similarly, when the squares of the differences between the U and V values are summed up, the sums become 8694 and 3072, respectively.

Once the total values corresponding to the Y, U, and V values are obtained in this way, they are compared with a preset reference value, and if each value is below the reference value, it is determined that the reference code and the specific target code match. The reference value can be set taking into consideration the application, etc., of the level of similarity that is deemed to be a "match." For example, a reference value equivalent to a match rate of 97% or more can be set.

The difference between the Y, U, and V values is further squared to make it easier to detect the difference between the reference code and the specific target code. Depending on the application, squaring may not be necessary, or conversely, the exponent may be increased further, such as by cubes.

Next, a method of relative comparison will be described. First, for the reference code, a value Ya12 is calculated by squaring the difference in Y value between the sampling range of row 1, column 1 and the sampling range of row 1, column 2. In the illustrated example, Ya12 = (202 - 195) ² = 49. Similarly, values Ya13, Ya14, ... Ya19 are calculated by squaring the difference in Y value between the sampling range of row 1, column 1 and the eight sampling ranges thereafter.

Next, a value Ya23 is calculated by squaring the difference in Y values between the sampling range of row 1, column 2 and the sampling range of row 1, column 3. In the example shown, Ya23 = (195 - 213) ² = 324. Similarly, values Ya24, Ya25, ..., Ya29 are calculated by squaring the differences in Y values between the sampling range of row 1, column 2 and the seven sampling ranges thereafter. Similarly, values Y34, Y35, ..., Y49 are calculated by squaring the differences in Y values between the sampling range of row 1, column 3 and the subsequent sampling ranges.

Similarly, calculate the squared difference in Y values between the sampling range of row 2, column 1 and the sampling range thereafter, the sampling range of row 2, column 2 and the sampling range thereafter, the sampling range of row 2, column 3 and the sampling range thereafter, the sampling range of row 3, column 1 and the sampling range thereafter, and the sampling range of row 3, column 2 and the sampling range thereafter.

Then, all the squared values of the differences in the Y values thus found are summed up. In the example shown, the total is 47,502. If the same calculation is performed for the U and V values, the total for the U values is 73,256, and the total for the V values is 0.

Next, if we perform the same calculations for the specific target codes as above, the total for the Y values is 70,898, the total for the V values is 58,104, and the total for the U values is 18,272.

Then, when calculating the difference between the total values of YUV between the reference code and the specific target code, the results are ΔY = 23396, ΔU = 15152, and ΔV = 18272.

Once the differences in the total values corresponding to the Y, U, and V values are obtained in this way, they are compared with a preset reference value, and if each value is below the reference value, it is determined that the reference code and the specific target code match. The reference value can be set taking into consideration the application, etc., of the level of similarity that is deemed to be a "match," and can be set to, for example, a reference value equivalent to a match rate of 97% or more.

As with simple comparison, the difference between the Y, U, and V values is further squared to make it easier to detect the difference between the reference code and the specific target code. Depending on the application, squaring may not be necessary, or conversely, a larger exponent such as cubed may be used. Also, in the above example of relative comparison, the difference between a certain sampling range and the sampling range thereafter was found, but the difference may also be found by exhaustively comparing each sampling range with all other sampling ranges.

The code according to the second embodiment as described above can be applied to the various application examples described in the first embodiment above. That is, although there is a difference between the method of generating the code in the first and second embodiments, each application example relates to a method of utilizing the generated code and is not affected by the method of generating the code.

Third Embodiment
Fig. 20 is a diagram for explaining the coding method of the third embodiment. Here, sound data is taken as an example of data to be coded (target data) for explanation. Note that the configurations of the code generation device and user device are the same as those of the first embodiment (see Figs. 6 and 7), and only the information processing method relating to code generation and matching is different. Here, the description of the device configuration is omitted, and the information processing method for code generation and matching will be described in detail.

Generally, the three elements of sound are loudness, pitch, and timbre. Sound data coding is not a technique for reproducing the sound itself, but rather for the purpose of recognizing and identifying sounds by analyzing sound data in real time, for example. In this embodiment, coding is performed with a focus on pitch, one of the three elements of sound.

Figure 20 (A) is an example of a waveform diagram in which a discrete Fourier transform is performed on a sound picked up by a microphone to determine the change in frequency over time. If necessary, the fundamental frequency may be calculated by cepstrum analysis. For such a waveform diagram, for example, a sampling interval Q is set to 0.2 seconds, and a frequency is extracted for each sampling interval. The extracted frequency value is then coded and recorded. At this time, related data such as the sampling interval Q, the sampling time (2.0 seconds in the illustrated example), and the title of the song are added and recorded. The sound data to be coded may be obtained in real time by picking up sound from a microphone, or may be data that has been picked up and saved in advance. If the saved sound data is in a compressed format such as mp3, it is appropriately converted to an uncompressed format such as WAVE format and coded.

In the above information processing, data division for each sampling interval is performed by the sampling processing unit 20, frequency calculation is performed by the feature value calculation unit 21, coding is performed by the encoding processing unit 23, and the obtained code and related information are recorded by the code recording processing unit 24.

Figure 20 (B) is a diagram showing an example of coding. It shows an example of coding performed every 0.2 seconds based on the waveform of the sound data shown in Figure 20 (A). For example, when the acquisition timing is 0.2 seconds, the frequency is extracted as 1046.50 Hz, and the corresponding code (indicated as "CEC" in the figure) is obtained as 0416. This note corresponds to "C6" in the scale name and 64 in the keyboard symbol (in the case of an 88-key keyboard). The same applies to other acquisition timings. In this example, the audible range is assumed to be 20 Hz to 20,000 Hz when the frequency is extracted, and the decimal points are truncated when coding. For example, in the case of 20.2 Hz, the value after truncation is "20", which becomes "14" when coded in hexadecimal. Similarly, in the case of 261.62 Hz, the value after truncation is "261", which becomes "105" when coded in hexadecimal. Similarly, for example, 20000.0 Hz, the value after truncation is "20000", which becomes "4e20" when coded in hexadecimal. Therefore, the code generated in this embodiment can be expressed in a maximum of four hexadecimal characters, so the data size is a maximum of 2 bytes.

21 is a diagram for explaining the matching method using the chords of the third embodiment. As shown in FIG. 21(A), the chords corresponding to the pitch of a part (2000 ms) of a certain song (Twilight Message) are obtained in advance by setting the sampling interval Q to 0.2 seconds, and are recorded as a reference chord group. In the figure, for the sake of simplicity, the chords are represented by the alphabets A to G, and it is assumed that the contents of the chords represented by the same alphabet match. In response to this, the identification target data of each song given as real time (or sound data) is coded and matched with the reference data group. In the example shown in the figure, there is no part in each of the songs (1) and (3) where the sequence of the chords matches the reference data group, so it can be determined that the song is not a song identified by the reference data group. On the other hand, there is a part in the song (2) where the sequence of the chords matches the reference data group, so it can be determined that the song (2) is a song (Twilight Message) corresponding to the reference data group. Regarding the match/mismatch between the codes at each sampling interval Q, the codes may be judged as "matching" if they are completely the same, or may be judged as "matching" if they are within a certain error range (for example, an error range equivalent to ±10 Hz).

In the above information processing, data division for each sampling interval is performed by the sampling processing unit 60, frequency calculation is performed by the feature value calculation unit 61, coding is performed by the encoding processing unit 63, and the obtained code is matched with the reference data group by the matching processing unit 64.

Here, when encoding sound data, it is more preferable to set the sampling interval taking into consideration the tempo, especially when the target is sound data of a song. For example, in the case of a song with a tempo (BPM) of 120, 120 quarter notes (beats) are included in one minute. In this case, there are two beats per second, and the interval between one beat is 0.5 seconds. In this case, by setting (adjusting) the sampling interval to match the tempo of the sound data, encoding corresponding to the timing of the notes is possible. Specifically, the sampling interval is Q, the tempo is B, and the length of the beat of the target note is X. Here, the length of the beat of the note is set, for example, as X = 4 for a quarter note and X = 8 for an eighth note. In this case, as shown in FIG. 21(B), the sampling interval Q can be expressed as Q = (1/((B/60) x (X/4))) x 1000. By setting the sampling interval Q using this relational expression and encoding the sound data, the accuracy of determining the similarity of the songs is further improved. The lower column of Figure 21 (B) shows examples of specific values for the preferred sampling interval Q.

By encoding sound data in this way, it is possible to realize controls such as displaying the title of a song playing in space on the user device 2, or displaying AR content (augmented reality content), videos, and still images that match the song on the user device 2. Furthermore, with regard to the issue of so-called music plagiarism, it is also possible to present similarity or non-similarity as an objective judgment rather than a subjective human judgment. Furthermore, assuming that the heart sounds of living organisms are generally different, even the heart sounds of twins are different, encoding heart sounds makes it possible to perform person authentication based on heart sounds (heart sound authentication). For example, it will be possible to create unprecedented use cases, such as person authentication at gates at airports, etc.

According to each of the above-described embodiments, a new technology is provided that encodes target data such as images and audio with a small amount of data. This also provides a new technology that makes it possible to match specific target data with reference data with a low load and in a short processing time.

That is, according to each embodiment, for example, an image captured by a camera can be divided into an arbitrary number of parts, and a unique code can be generated and recorded for each of the divided blocks (sampling ranges) from information such as light source, brightness, and color, and this code can be used to perform high-speed (or real-time) comparison, collation, and authentication. By using this unique code, it is possible to transmit invisible information such as heat sources, sound, and X-rays, as well as information from the five senses, with a smaller data size. As shown in the conceptual diagram in FIG. 22, by adding a unique code according to each embodiment to what is currently established and recognized as data, such as spatial information, moving images, still images, temperature, and sound, not only can it be linked to the real world, but new ways of using data can be created, and further added value and methods of using data that have already been recorded can be given. For example, in addition to GPS information (location information), direction, angle, temperature, pressure, and the like, various accompanying information such as source and URL already exists in big data. However, when comparing and collating with the real world in real time, the "situation," "state," and "spatial data" necessary for judgment are missing, and the necessary linking cannot be made. In response to this, the code according to each embodiment can be used to create a link with reality called "Earth Data," which can act as a "key" or trigger for the link between the real world and the system. Because the data size of the code according to each embodiment is very small, a new concept called "byte chain" is provided.

The present invention is not limited to the contents of the above-mentioned embodiments, and various modifications can be made within the scope of the gist of the present invention. The use and application of the code generated by each embodiment is not limited to the above. The numerical values given in each embodiment are merely examples and are not limited to these. In the first embodiment, the relative difference between each sampling range is calculated, but this may be omitted and the average brightness value of each sampling range may be concatenated as character string data.

In addition, in each of the above-mentioned embodiments, the code is generated by directly concatenating the character string data obtained from the average values or the relative differences, but the form of "concatenation" in the present invention is not limited to this. For example, the character string data obtained by multiplying each average value or each relative difference by a certain coefficient may be concatenated. Furthermore, the character string data may be concatenated in a form in which specific data (such as a delimiter) is interposed between the character string data corresponding to each average value, etc. Alternatively, character string data corresponding to each average value, etc. may be converted into other character string data under a certain rule and concatenated. In other words, any form of concatenation is included in the concept of "concatenation" in the present invention as long as it does not destroy or can restore the relative relationships between the average values, etc.

10, 50: Information processing unit, 11, 51: Camera, 12, 52: Microphone, 13, 53: Storage device, 14, 54: Operating device, 15, 55: Display device, 20, 60: Sampling processing unit, 21, 61: Feature value calculation unit, 23, 63: Encoding processing unit, 24: Code recording processing unit, 64: Matching processing unit

Claims (3)

A method for generating and verifying a code according to the contents of target data using an information processing device, comprising:
A first step of obtaining a reference code determined according to reference data;
a second step of generating a specific target code corresponding to the specific target data;
a third step of comparing said reference code with said specific target code to determine a match/mismatch therebetween;
Including,
The reference code in the first step is
Dividing the reference data into a plurality of first sampling ranges;
a step of calculating a first average value of at least one type of first element data among one or more types of first element data included in each of the plurality of first sampling ranges, each of which is expressed by a numerical value, for each of the first sampling ranges;
determining a first relative difference for each of the first sampling ranges by dividing the first average value for each of the first sampling ranges by a sum of the first average values for each of the first sampling ranges;
generating the reference code corresponding to the reference data by concatenating, as character string data, the first relative difference for each of the first sampling ranges or a predetermined number of digits from the first relative difference;
It was previously sought through
The identified target code in the second step is
Dividing the identified target data into a plurality of second sampling ranges set in the same manner as each of the first sampling ranges;
a step of calculating a second average value of at least one type of second element data among one or more types of second element data included in each of the plurality of second sampling ranges, each of which is expressed by a numerical value and which is the same type as the first element data, for each of the second sampling ranges;
determining a second relative difference for each of the second sampling ranges by dividing the second average value for each of the second sampling ranges by a sum of the second average values for each of the second sampling ranges;
generating the specific target code corresponding to the specific target data by concatenating, as character string data, the second relative difference for each of the second sampling ranges or a predetermined number of digits from the second relative difference;
It is generated through
The third step is
a step of calculating a difference between the first relative difference or a predetermined number of digits from the most significant digit for each of the first sampling ranges obtained based on the reference code and the second relative difference or a predetermined number of digits from the most significant digit for each of the second sampling ranges obtained based on the specific target code, between each of the first sampling ranges and each of the second sampling ranges;
determining that the reference code and the specific target code match when each of the differences falls within a predetermined allowable range;
A method for matching data. the number of the first sampling ranges and the number of the second sampling ranges are equal to or greater than 64;
The data matching method according to claim 1 . The reference data and the identification target data include still image data, moving image data, or sound data.
The data matching method according to claim 1 .

Images