JP7649264B2 - Three-dimensional shape measurement system and machine tool system - Google Patents

Description

This specification discloses a three-dimensional shape measurement system that measures the three-dimensional shape of an object based on image data obtained by capturing an image of the object, and a machine tool system equipped with the same.

In recent years, technology has been proposed for measuring the three-dimensional shape of an object based on an image obtained by capturing an image of the object. For example, Patent Document 1 discloses a processing device equipped with a light irradiation unit that irradiates a line of light onto the object, and an imaging unit that captures an image of the line of light reflected by the object. The machine tool in Patent Document 1 measures the three-dimensional shape of the surface of the object based on the image captured of the line of light.

JP 2019-69486 A

In order to measure the shape of an object based on an image, it is necessary to set the imaging conditions appropriately so that the surface shape of the object can be grasped from the image. For example, when light is strongly reflected from the object, blown-out highlights occur in the image due to halation. In the areas where such blown-out highlights occur, the surface shape of the object cannot be grasped, and as a result, the shape of the object cannot be measured. Also, when the amount of light irradiated on the object is insufficient and the brightness of the image is low, the surface shape of the object cannot be grasped and the shape of the object cannot be measured.

Therefore, in order to measure the shape of an object based on an image, it is necessary to appropriately set the imaging conditions, such as the camera gain. Here, the imaging conditions suitable for shape measurement differ depending on the material and surface properties of the object. For example, an object made of metal is more likely to reflect light and produce whiteouts than an object made of resin, etc. Also, even if the material is the same, an object with a small surface roughness is more likely to reflect light than an object with a large surface roughness. Therefore, in order to appropriately measure the shape, it is necessary to change the imaging conditions depending on the material and surface properties of the object.

Conventionally, these imaging conditions were often set by the operator, but it was difficult for the operator to identify appropriate imaging conditions. As a result, it was difficult for conventional 3D shape measurement systems to properly measure the shapes of various types of objects using simple procedures.

Therefore, this specification discloses a three-dimensional shape measurement system and a machine tool system that can appropriately measure the shapes of various types of objects using simpler procedures.

The three-dimensional shape measurement system disclosed in this specification includes an imaging unit having one or more cameras that capture images of an object, a storage device that stores, as condition information, imaging conditions for measurement imaging of an object to measure the shape of the object for each of a plurality of combinations of the object's material and surface properties, and a measurement controller that controls the operation of the imaging unit, and the measurement controller identifies the material and surface properties of the object, identifies the imaging conditions corresponding to the identified material and surface properties of the object based on the condition information, causes the imaging unit to perform the measurement imaging under the identified imaging conditions, and measures the shape of the object based on the obtained measurement images.

In this case, the measurement controller may cause the imaging unit to image the object prior to the measurement imaging, and identify at least one of the material and surface properties of the object based on the preliminary image thus obtained.

The storage device may further store a learning model that uses the preliminary image as an input and outputs at least one of the material and surface properties of the object, and the measurement controller may identify at least one of the material and surface properties of the object based on the learning model.

The imaging unit may be provided in a machine tool, and may capture an image of a workpiece being machined by the machine tool as the target object, and the measurement controller may identify at least one of the material and surface properties of the target object based on at least one of a machining program for the workpiece and a machining drawing for the workpiece.

The measurement controller may also acquire information indicating at least one of the machining program and the machining drawing from a numerical control device of the machine tool.

In addition, at least one of the material and surface properties of the object may be specified by the operator.

The imaging unit may further have one or more light sources that irradiate imaging light onto the object, and the measurement controller may perform one or more basic imaging operations to image the object in order to obtain one of the measurement images, and the imaging conditions may include at least one of the number of times the basic imaging operations are performed to obtain one of the measurement images, the shutter speed of the camera in each of the one or more basic imaging operations, the gain of the camera in each of the one or more basic imaging operations, and the luminance distribution of the imaging light in each of the one or more basic imaging operations.

The measurement controller may also generate point cloud data of the object based on the measurement image, and generate three-dimensional data of the object from the point cloud data.

The machine tool system disclosed in this specification includes the above-mentioned three-dimensional shape measurement system and a machine tool that performs machining on the object, and is characterized in that the machine tool performs at least one of the following based on the three-dimensional data generated by the measurement controller: checking for interference between a tool and the object, generating a path for the tool, determining machining accuracy, determining whether the shape of the object matches a reference shape, and determining whether the object is located at a predetermined position.

The technology disclosed in this specification allows for easy and proper shape measurement of various types of objects.

FIG. 1 is an image diagram showing a configuration of a machine tool system. FIG. 1 is a block diagram showing a configuration of a shape measurement system. 1 is an image diagram showing the principle of three-dimensional shape measurement of an object by a shape measurement system. FIG. 13 is an image diagram showing how an HDR image is generated. FIG. 11 is a diagram illustrating an example of condition information. FIG. 1 is a diagram showing an example of a processing drawing. FIG. 4 is a diagram showing an example of a machining program. 11 is a diagram showing an example of a correspondence relationship between a value of a common variable and a material and a surface property. FIG. FIG. 13 is a diagram showing an example of a correspondence relationship between a sequence name and a material and a surface property. FIG. 11 is a diagram showing another example of a machining program. 10 is a flowchart showing a process flow of the shape measurement system. 10 is a flowchart showing a process flow of the shape measurement system.

Below, the configuration of a three-dimensional shape measurement system 12 (hereinafter abbreviated as "shape measurement system 12") and a machine tool system 10 incorporating the shape measurement system 12 will be described with reference to the drawings. Figure 1 is an image diagram showing the configuration of the machine tool system 10.

As shown in FIG. 1, the machine tool system 10 includes a machine tool 14 and a shape measurement system 12. The machine tool 14 performs a predetermined process on a workpiece to manufacture a processed product. The type of the machine tool 14 is not particularly limited, but for example, the machine tool 14 may be a metal processing machine that performs metal processing (e.g., cutting, pressing, etc.) on a metal workpiece. In FIG. 1, the machine tool 14 is a lathe having a spindle 14a and a tool rest 14b. In the following description, the workpiece handled by the machine tool 14 is treated as the object 100 for shape measurement. However, the object 100 for shape measurement is not limited to a workpiece, and may be other members, such as tools or jigs.

The machine tool 14 has a numerical control device 20. The numerical control device 20 is a device that analyzes a machining program 66 (also called an "NC program") and commands the tool path for the workpiece, the steps of the work required for machining, etc., using numerical information consisting of numbers and codes to operate the machine tool 14. The numerical control device 20 checks for interference between the tool and the workpiece, generates tool paths, judges the machining accuracy, judges whether the shape of the object 100 matches a reference shape, judges whether the object 100 is in a specified position, etc., based on three-dimensional data 69 (e.g., CAD data, etc.) that indicates the shape of the workpiece (i.e., the object 100). The numerical control device 20 is physically a computer having a processor and memory. The machine tool 14 is also provided with an operation panel 22 that presents information to the operator and accepts commands from the operator.

Furthermore, the numerical control device 20 stores data necessary for machining the target object 100, such as a machining program 66 and a machining drawing 64. The numerical control device 20 can communicate with the measurement controller 30 by wired or wireless communication, and transmits the machining program 66, etc. to the measurement controller 30 as necessary. Note that, although the numerical control device 20 is configured to store the machining program 66, etc. in this example, the machining program 66, etc. may be stored in another external storage device that can communicate with the measurement controller 30. For example, a cloud data server or the like may store the machining program 66, etc., and transmit it to the measurement controller 30.

The shape measurement system 12 measures the shape of the object 100, and includes a measurement controller 30 and an imaging unit 50. The imaging unit 50 captures an image of the object 100. The image captured by the imaging unit 50 is sent to the measurement controller 30 as a measurement image 60 or a preliminary image 62, which will be described later.

The measurement controller 30 specifies the imaging conditions 56 for capturing the measurement image 60, and instructs the imaging unit 50 to capture the image under the specified imaging conditions 56. Note that in the following, the symbols for the imaging conditions will be omitted. The measurement controller 30 also generates point cloud data 74 of the object 100 based on the measurement image 60 sent from the imaging unit 50, and generates three-dimensional data 69 indicating the three-dimensional shape of the object 100 based on this point cloud data 74. The generated three-dimensional data 69 is sent to the numerical control device 20.

Next, the configuration of the shape measurement system 12 will be described in more detail. FIG. 2 is a block diagram showing the configuration of the shape measurement system 12. As shown in FIG. 2 and as described above, the shape measurement system 12 has a measurement controller 30 and an imaging unit 50.

The measurement controller 30 is physically a computer having a processor 32, a memory 34, a communication I/F 36, a UI device 38, and a storage device 40. This "computer" also includes a microcontroller in which a computer system is incorporated into a single integrated circuit. The processor 32 refers to a processor in a broad sense, and includes a general-purpose processor (e.g., CPU: Central Processing Unit, etc.) and a dedicated processor (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operation of the measurement controller 30 described below may not only be performed by one processor, but may also be performed by the cooperation of multiple processors located at physically separate locations. Similarly, the memory 34 does not have to be a single physical element, but may be composed of multiple memories located at physically separate locations. Furthermore, the memory 34 may include, for example, a semiconductor memory (e.g., RAM, ROM, etc.).

The communication I/F 36 transmits and receives data to and from other external electronic devices through wired or wireless communication. The communication I/F 36 communicates with, for example, the numerical control device 20 and the imaging unit 50. The UI device 38 presents various information to the operator and accepts instructions from the operator. The UI device 38 has, for example, output devices such as a display and a speaker, and input devices such as a keyboard, a mouse, a microphone, and a touch panel. In this example, the UI device 38 is described as a component of the measurement controller 30, but part or all of the UI device 38 may be configured as a completely separate device from the measurement controller 30. For example, some functions of the UI device 38 may be realized by a user interface of an information terminal (e.g., a smartphone) owned by the operator and capable of communicating with the measurement controller 30.

The storage device 40 stores various data required for shape measurement, and is physically configured, for example, as a hard disk drive, a solid state drive, or a combination of these. Note that in this example, the storage device 40 is described as a component of the measurement controller 30, but part or all of the storage device 40 may be configured as a separate device completely different from the measurement controller 30. For example, the storage device 40 may be a storage device provided in a cloud server capable of communicating with the measurement controller 30.

The storage device 40 stores condition information 42, reference data 44, and a learning model 45. The condition information 42 is data in which the imaging conditions for measurement imaging are recorded separately for each of a plurality of combinations of the material and surface properties of the object 100. The specific contents of this condition information 42 will be described later. The reference data 44 is a plurality of image data that are compared with the preliminary image 62, which will be described later, in order to identify the material and surface properties of the object 100. The specific contents of this reference data 44 will also be described later. The learning model 45 will also be described later.

The imaging unit 50 captures an image of the object 100 present in the machining chamber of the machine tool 14. As shown in FIG. 2, the imaging unit 50 has a camera 52 and a projector 54. The camera 52 captures an image of the object 100 and generates image data. The image data obtained by the camera 52 is sent to the measurement controller 30. The imaging conditions for capturing an image of the object 100, such as the number of images taken and the shutter speed, are instructed by the measurement controller 30.

The projector 54 functions as a light source 53 that irradiates the object 100 with imaging light. The projector 54 (light source 53) in this example can freely change the luminance distribution of the imaging light that irradiates the object 100. The luminance distribution of this imaging light is instructed by the measurement controller 30.

Next, the principle of three-dimensional shape measurement of the object 100 by the shape measurement system 12 will be described with reference to FIG. 3. When measuring the three-dimensional shape of the object 100, the measurement controller 30 drives the imaging unit 50 to obtain a measurement image 60. At this time, the relative orientation of the object 100 with respect to the camera 52 is changed and imaging is performed multiple times so that a measurement image 60 can be obtained for all surfaces of the object 100. Note that in order to change the relative orientation of the object 100 with respect to the camera 52, the object 100 may be moved or the camera 52 may be moved. For example, the orientation of the object 100 with respect to the camera 52 may be changed by driving the main shaft 14a that holds the object 100.

The measurement controller 30 generates point cloud data 74 of the object 100 based on the obtained measurement image 60. The point cloud data 74 is also called a "point cloud" and is data that represents the surface shape of the object 100 with multiple points having X, Y, and Z coordinate values. Such point cloud data 74 can be generated using, for example, a stereo method, a phase shift method, a light section method, etc. The stereo method is a technique in which the camera 52 is moved or two cameras 52 are used to capture the object 100 from different positions, corresponding points are searched from the parallax images, and the height is obtained by triangulation. The phase shift method is a technique in which a sine wave is projected onto the object 100 by the projector 54 while shifting the phase, and the phase is obtained from the luminance change of the image obtained by capturing the image multiple times. Furthermore, the light section method is a technique in which a line light is projected onto the object 100, which is captured by the camera 52, and the shape of the object 100 is measured based on the obtained image. These techniques are all conventionally known, so detailed explanations will be omitted here. Once the point cloud data 74 is obtained, the measurement controller 30 generates three-dimensional data 69 of the object 100 based on this point cloud data 74. The generated three-dimensional data 69 is sent to the numerical control device 20 or a computer designated by the operator as necessary.

In order to properly generate the point cloud data 74, the object 100 must be properly captured in the measurement image 60. However, when the object 100 is simply captured by the camera 52, blackout or whiteout may occur. Blackout is a phenomenon in which the dark parts of an image become completely black, and whiteout is a phenomenon in which the bright parts of an image become completely white. Regardless of whether blackout or whiteout occurs, the measurement controller 30 cannot properly determine the shape of the object 100. Therefore, in this example, in order to obtain one measurement image 60, the object 100 is captured once or more under predetermined shooting conditions. Hereinafter, the imaging performed to obtain this one measurement image 60 is called "basic imaging", and the image obtained by the basic imaging is called "basic image 70". When basic imaging is performed only once to obtain one measurement image 60, the measurement controller 30 treats the obtained one basic image 70 as the measurement image 60. If basic imaging is performed two or more times, the measurement controller 30 generates a high dynamic range image 72 (hereinafter referred to as "HDR image 72") from two or more basic images 70, and treats this HDR image 72 as the measurement image 60.

The HDR image 72 is an image obtained by synthesizing two or more basic images 70 obtained by capturing the object 100 two or more times under different capturing conditions so as to have a high luminance range. FIG. 4 is a diagram showing how the HDR image 72 is generated. In the example of FIG. 4, the object 100 is illuminated by a light (not shown) located on the rear side, and has a protrusion 102 on the front side. Also, in FIG. 4, the first basic image 70f is an image captured by adjusting the exposure to the protrusion 102, and the second basic image 70s is an image captured by adjusting the exposure to the edge 104 on the rear side. In this case, in the first basic image 70f, the entire image is bright, so the shape of the protrusion 102 can be grasped, but the shape of the edge 104 on the rear side cannot be grasped due to whiteout. In the second basic image 70s, the entire image is dark, so the shape of the edge 104 on the rear side can be grasped, but the shape of the protrusion 102 cannot be grasped due to blackout.

The HDR image 72 is obtained by combining multiple basic images 70 in such a way that the brightness range is increased. In the example of FIG. 4, data around the protrusion 102 of the first basic image 70f and data around the back edge 104 of the second basic image 70s are combined to generate a single HDR image 72. Based on such an HDR image 72, the shape of the object 100 can be appropriately grasped.

Incidentally, the imaging conditions (e.g., the number of basic imaging times, the shutter speed of the camera 52, the luminance distribution of the imaging light, etc.) required to obtain an appropriate measurement image 60 vary depending on the material and surface properties of the object 100. For example, aluminum generally has a whitish silver color compared to steel materials such as S45C, and is therefore more likely to reflect light and cause highlight blowout. Also, even if the material is the same, an object 100 with a rough surface roughness is less likely to reflect light and cause highlight blowout than an object 100 with a fine surface roughness. Therefore, in order to obtain an appropriate HDR image 72, it is necessary to change the imaging conditions depending on the material and surface properties of the object 100.

The measurement controller 30 in this example therefore stores in advance appropriate imaging conditions for each of multiple combinations of the material and surface properties of the object 100 as condition information 42. Then, when measuring the shape of the object 100, the material and surface properties of the object 100 are identified, imaging conditions corresponding to the identified material and surface properties are identified, and the object 100 is imaged under the identified imaging conditions. The identification of these imaging conditions will be described below.

Figure 5 is a diagram showing an example of condition information 42 stored in storage device 40. As shown in Figure 5, in condition information 42, the imaging conditions are managed separately for each of a plurality of combinations of the material and surface properties of object 100. In this example, the surface properties are expressed by the stage of processing applied to object 100, i.e., "unprocessed," "rough processing," "semi-finished," "finished," "mirror finish," etc. However, the expressions of surface properties given here are only examples, and surface properties may be expressed in other forms. For example, surface properties may be expressed by the surface roughness, diffuse reflectance, surface treatment state, etc. of object 100.

In the example of FIG. 5, the imaging conditions include the number of basic imaging operations performed to obtain one measurement image 60, and the shutter speed, camera gain, and luminance distribution of the imaging light for each basic imaging operation. However, the parameters listed here are merely examples and may be changed as appropriate. For example, the imaging conditions may include only the number of basic imaging operations and the camera gain, and may not include the shutter speed and luminance distribution of the imaging light. The numerical values recorded in such condition information 42 are obtained in advance by experiment or simulation. Furthermore, the operator may add imaging conditions for new materials and surface properties, or modify the recorded imaging conditions, as necessary.

The measurement controller 30 identifies the material and surface properties of the object 100, and compares them with the condition information 42 to identify the imaging conditions corresponding to the material and surface properties of the object 100. For example, if the material of the object 100 is "SS400" and the surface properties of the object 100 are "unfinished", the measurement controller 30 performs basic imaging of the object 100 once to measure the shape of the object 100. At this time, the shutter speed of the camera 52 is "20", the camera gain is "2", and the luminance distribution of the imaging light is "R, G, B = 164, 128, 128". Also, if the material of the object 100 is "SS400" and the surface properties of the object 100 are "finished", the measurement controller 30 performs basic imaging of the object 100 twice to measure the shape of the object 100. In the first basic imaging, the shutter speed of the camera 52 is "10", the camera gain is "1", and the luminance distribution of the imaging light is "R, G, B = 64, 64, 96". In the second basic imaging, the shutter speed of the camera 52 is "12", the camera gain is "1", and the luminance distribution of the imaging light is "R, G, B = 128, 128, 128".

Next, the identification of the material and surface properties of the object 100 will be described. In this example, the measurement controller 30 identifies at least one of the material and surface properties of the object 100 based on at least one of the processing drawing 64 and the processing program 66, which record the processing details of the object 100. If the material and surface properties cannot be identified using the processing drawing 64 and the processing program 66, the measurement controller 30 captures an image of the object 100 under standard conditions to obtain a preliminary image 62, and analyzes this preliminary image 62 to identify at least one of the material and surface properties of the object 100. If the material and surface properties cannot be identified using the preliminary image 62, the measurement controller 30 requests the operator to input the material and surface properties of the object 100. This will be described in order below.

First, a technique for identifying at least one of the material and the surface texture based on at least one of the processing drawing 64 and the processing program 66 will be described. When identifying the material and the surface texture, the measurement controller 30 acquires the processing drawing 64 and the processing program 66 of the object 100. FIG. 6 is a diagram showing an example of the processing drawing 64. As shown in FIG. 6, the processing drawing 64 usually includes the product name and the material of the object 100 in addition to a diagram showing the shape of the object 100 after processing is completed. The measurement controller 30 performs image analysis processing such as OCR on the processing drawing 64 to extract the text described in the processing drawing 64. Then, the measurement controller 30 identifies at least one of the material and the surface texture of the object 100 based on the extracted text. For example, if there is text indicating a specific material in the extracted text, the measurement controller 30 determines that the object 100 is made of the specific material. Similarly, if there is text indicating a specific surface texture in the extracted text, the measurement controller 30 determines that the object 100 is made of the specific surface texture. Therefore, for example, if the extracted text contains the text "S45C" and "rough machining," the measurement controller 30 determines that the material of the object 100 is "S45C" and that the surface quality of the object 100 is "rough machining."

In another embodiment, the measurement controller 30 may identify at least one of the material and surface properties of the target object 100 based on the machining program 66. The machining program 66 describes commands for causing the machine tool 14 to perform intended operations, and is also called an "NC program." When identifying the material, etc. based on the machining program 66, the operator embeds information indicating the material, etc. in the machining program 66 according to predetermined rules.

For example, in the machining program 66, the portion enclosed in parentheses usually indicates a comment, and the comment is not used to control the machine tool 14. Therefore, if text indicating at least one of the material and surface texture is written as a comment in the machining program 66, it may be identified as at least one of the material and surface texture of the object 100. For example, in the case of the machining program 66 shown in FIG. 7, "S45C, ROUGH" is written as a comment in the third line. Of these, "S45C" is text indicating "S45C", which is a type of material, and "ROUGH" is text indicating "rough machining", which is a type of surface texture. Therefore, in this case, the measurement controller 30 identifies the material of the object 100 as "S45C" and the surface texture of the object 100 as "rough machining".

In another embodiment, at least one of the material and the surface texture may be represented by the value of a variable or command used in the machining program 66. In this case, the measurement controller 30 stores in advance the correspondence between the value of the variable or command and the material and the surface texture. FIG. 8 is an example of data showing such a correspondence. The table 80a on the left side of FIG. 8 is a material correspondence table 80a that records the correspondence between the value of the common variable "VC101" and the material, and the table 80b on the right side is a surface texture correspondence table 80b that records the correspondence between the value of the common variable "VC102" and the surface texture. The measurement controller 30 stores such correspondence tables 80a and 80b in the storage device 40 in advance. In the example of FIG. 8, when the value of the common variable "VC101" is "10", it represents the material "S45C", and when the value of the common variable "VC102" is "20", it represents the surface texture "rough machining". Therefore, if the machining program 66 contains the statements "VC101=10;" and "VC102=20;", the measurement controller 30 will identify the material of the object 100 as "S45C" and the surface condition as "rough machining".

Also, instead of using variables, a dedicated command code may be created and the material or surface texture may be assigned to the value of the command code. For example, a code "MP" may be created to command the material, and if the MP value is "1", it may be set to "S45C", if it is "2", it may be set to "SS400", etc. Similarly, a code "MQ" may be created to command the surface texture, and if the MQ value is "0", it may be set to "unmachined", if it is "1", it may be set to "roughly machined", etc. In this case, if the machining program 66 contains the statement "MP 2; MQ 0;", the measurement controller 30 will determine that the material is "SS400" and the surface texture is "unmachined".

As another form, at least one of the material and surface properties may be represented using a sequence name (also called a "sequence number") written in the machining program 66. The sequence name indicates the relative position of a block (line) in the machining program 66, and is written at the beginning of any line. Typically, the sequence name is written as "N + alphanumeric characters". For example, if there is a line "N100;" in the machining program 66, that line represents the sequence name "N100".

When the sequence name indicates at least one of the material and the surface property, the measurement controller 30 stores in advance the correspondence between the sequence name and at least one of the material and the surface property. FIG. 9 is a diagram showing an example of such a correspondence. In the case of FIG. 9, the second character of the sequence name indicates a shape measurement command, the third character indicates the material, and the fourth character indicates the surface property. In this case, if the second character of the sequence name is "Z", the measurement controller 30 determines that the third character of the sequence name indicates the material and the fourth character indicates the surface property. Also, in the example of FIG. 9, the third character "S" indicates the material "S45C", and the fourth character "A" indicates the surface property "rough machining". Therefore, in this case, if the processing program 66 contains the description "NZSA;", the measurement controller 30 identifies the material of the object 100 as "S45C" and the surface property as "rough machining".

As another example, the surface texture may be determined based on the tool specified in the machining program 66. In this case, the measurement controller 30 stores in advance the correspondence between the tool number and the surface texture. For example, the measurement controller 30 stores in advance the tool number and the corresponding machining content (and thus the surface texture), such as tool number "01" being an end mill for rough machining and tool number "02" being an end mill for semi-finishing. In this case, if the machining program 66 contains the statement "T01; M06;", the measurement controller 30 determines the surface texture of the target object 100 as "rough machining".

In another embodiment, the surface texture may be specified based on the machining parameters specified in the machining program 66. In this case, the measurement controller 30 stores in advance the correspondence between the numerical range of the machining parameters and the surface texture. In this case, the machining parameters include, for example, the spindle speed, the cutting feed rate, the amount of cut, etc. The measurement controller 30 stores the numerical range of the spindle speed, the numerical range of the cutting feed rate, and the numerical range of the amount of cut for each type of surface texture. Then, the measurement controller 30 may specify the surface texture based on the command values of these machining parameters described in the machining program 66. For example, consider the case where there is a machining program 66 shown in FIG. 10. In the machining program 66, S is the spindle speed, F is the cutting feed rate, and PQ is a variable indicating the amount of cut defined by the operator. Also, M3 is a spindle normal rotation command, and G01 is a cutting feed command. Therefore, the machining program 66 in Figure 10 indicates that the machining is performed at a spindle speed of 4000 rpm, a cutting feed rate of 100 mm/min, and cutting 2 mm from a position of 100 mm in the Z direction. The measurement controller 30 checks these values against the numerical ranges of parameters stored in advance to identify the surface properties.

As another example, a file describing the material and surface properties may be created, and an access command to this file may be written in the machining program 66. For example, "MR" may be set as a command code indicating access to a file at a specified file path, and the material and surface properties may be written in text or binary format in the file. Then, if "MR;" is written in the machining program 66, the measurement controller 30 will access the file at the specified file path and identify the material and surface properties.

In addition, when the material and surface properties cannot be identified from the processing drawing 64 and the processing program 66, the material and surface properties of the object 100 may be identified based on the captured image of the object 100. That is, in this case, the measurement controller 30 stores a group of images of objects of various materials and surface properties as reference data 44 in advance. In addition, when it is desired to identify the material and surface properties of the object 100, the measurement controller 30 drives the imaging unit 50 to capture the object 100 and obtain a preliminary image 62. The imaging conditions for this preliminary image 62 are standard conditions. In other words, the imaging conditions for the preliminary image 62 are not changed according to the material and surface properties of the object 100.

The measurement controller 30 compares this preliminary image 62 with the image group of the reference data 44, and identifies an image similar to the preliminary image 62. The measurement controller 30 may then identify the material and surface properties of the target object 100 based on the material and surface properties of the object depicted in the identified image. Note that, for example, a technique such as pattern matching may be used in comparing the preliminary image 62 with the image group. Furthermore, if the material and surface properties cannot be identified from any of the processing drawing 64, processing program 66, and preliminary image 62, the measurement controller 30 may request the operator to input the material and surface properties.

In another embodiment, the measurement controller 30 may use the learning model 45 to identify the material and surface properties of the object 100. In this case, the measurement controller 30 stores the learning model 45, which has been generated in advance by machine learning, in the storage device 40 (see FIG. 2). The learning model 45 can be generated by supervised learning, in which images of an object whose material and surface properties are known (e.g., a group of images of the reference data 44 labeled with information on the material and surface properties) are input to a learning device as teacher data, and the learning device learns the correlation between the images and the material and surface properties. The method of generating the learning model 45 by supervised learning is publicly known, so a detailed description will be omitted here.

The measurement controller 30 can obtain the corresponding material and surface texture as an output by inputting the preliminary image 62 into the learning model 45. The output material and surface texture may be modified by the operator's input. In another embodiment, the learning model 45 may output a plurality of combinations of material and surface texture for one preliminary image 62. In this case, the measurement controller 30 presents the obtained plurality of combinations to the operator in order of likelihood. The combination selected and determined by the operator's input may be specified as the material and surface texture of the object 100. In this case, the learning model 45 may be updated by creating teacher data using the preliminary image 62 as input data and labeling the combination of material and surface texture modified or selected by the operator, and re-learning by the learning device. With such a configuration, the accuracy of estimating the material and surface texture can be improved. In the above description, both the material and surface texture are specified, but the configuration may be such that only one of the material and surface texture is specified.

Once the material and surface properties of the object 100 can be identified, as described above, the measurement controller 30 compares the identified material and surface properties with the condition information 42 to identify the imaging conditions for capturing the measurement image 60. Then, when the trigger for the shape measurement of the object 100 is turned ON, the shape measurement system 12 executes measurement imaging to capture the object 100 under the identified imaging conditions. Here, the trigger for the shape measurement may be an operation instruction from the operator. For example, when the operator operates the operation panel 22 of the numerical control device 20 or the UI device 38 of the measurement controller 30 to instruct "start shape measurement," measurement imaging of the object 100 may be executed.

As another embodiment, a trigger code may be written in the machining program 66. For example, a dedicated M code or G code that instructs the execution of three-dimensional shape measurement may be written in the machining program 66, and when the time comes for the machine tool 14 to execute this dedicated M code or G code, the shape measurement system 12 may execute the measurement imaging.

Next, the process flow of the shape measurement system 12 will be described with reference to Figs. 11 and 12. When the shape measurement system 12 is started, it first initializes the entire system (S10). Specifically, the measurement controller 30 initializes the parameter settings of the camera 52 and the projector 54.

Once the initialization is complete, the measurement controller 30 identifies the material and surface properties of the object 100 (S12 to S24). Specifically, the measurement controller 30 first communicates with the numerical control device 20 and attempts to acquire the machining drawing 64 and machining program 66 (S12). If the machining drawing 64 and machining program 66 are acquired (Yes in S12), the measurement controller 30 analyzes the machining drawing 64 and machining program 66 and identifies the material and surface properties of the object 100 (S16).

On the other hand, if the processing drawing 64 and processing program 66 cannot be acquired (No in S12), the measurement controller 30 captures an image of the object 100 with the imaging unit 50 and acquires a preliminary image 62 (S20). Next, the measurement controller 30 compares the obtained preliminary image 62 with the reference data 44 or inputs it into the learning model 45 to identify the material and surface properties of the object 100 (S22).

If the material and surface texture are obtained based on the processing drawing 64 or the preliminary image 62 (Yes in S18), the process proceeds to step S26. On the other hand, if at least one of the material and surface texture is not obtained (No in S18), the measurement controller 30 requests the operator to input at least one of the material and surface texture, thereby identifying the material and surface texture (S24).

If the material and surface properties can be identified, the measurement controller 30 checks the identified material and surface properties against the condition information 42. If the condition corresponding to the identified material, etc. is recorded in the condition information 42 (Yes in S26), the measurement controller 30 sets the corresponding condition as the imaging condition (S28). On the other hand, if the condition corresponding to the identified material, etc. is not recorded in the condition information 42 (No in S26), the measurement controller 30 requests the operator to newly register the imaging condition corresponding to the identified material, etc. In response to this request, if the operator newly registers the imaging condition (Yes in S30), the measurement controller 30 accepts this new registration (S32) and returns to step S26. On the other hand, if the operator does not newly register (No in S30), the measurement controller 30 requests the operator to input the imaging condition. Then, the condition input by the operator in response to this request is set as the imaging condition (S34).

Once the imaging conditions have been set, the measurement controller 30 monitors whether the trigger for measuring the shape of the target object 100 is ON (S36). As described above, the trigger for measuring the shape is an instruction from the operator or a specific code in the machining program 66. Therefore, for example, the measurement controller 30 may monitor whether a measurement instruction has been issued from the operator. In addition, the measurement controller 30 may periodically obtain the number of lines currently being executed in the machining program 66 from the numerical control device 20 as the progress of machining, and monitor whether a code instructing measurement has been executed based on this number of executed lines.

When the shape measurement trigger is ON (Yes in S36), the measurement controller 30 drives the imaging unit 50 to capture an image of the object 100 under the set imaging conditions and obtain a measurement image 60. Furthermore, the measurement controller 30 generates point cloud data 74 of the object 100 based on the obtained measurement image 60 (S38).

On the other hand, if the shape measurement trigger is not ON (No in S36) and the shape measurement system 12 is not OFF (No in S40), the measurement controller 30 further judges whether or not the imaging conditions need to be changed (S42). That is, if the object 100 itself is changed or another type of cutting is performed on the object 100 after the imaging conditions are set, the material and surface properties of the object 100 are likely to have changed. Therefore, the measurement controller 30 judges that the imaging conditions need to be changed when the code written in the machining program 66 in the numerical control device 20, which commands a tool change or a workpiece change, is executed, or when the machining program 66 executed by the numerical control device 20 itself is changed. If it is judged that the imaging conditions need to be changed (Yes in S42), the measurement controller 30 returns to step S12. On the other hand, if the imaging conditions do not need to be changed (No in S42), the measurement controller 30 returns to step S36.

As is clear from the above explanation, in this example, the imaging conditions for each of multiple combinations of the material and surface properties of the object 100 are stored as condition information 42. Furthermore, the measurement controller 30 identifies the material and surface properties of the object 100, identifies imaging conditions corresponding to the identified material and surface properties of the object 100 based on the condition information 42, and performs imaging of the object 100 under the identified imaging conditions. With this configuration, even if the material and surface properties of the object 100 are changed, the object 100 can be imaged under appropriate conditions. As a result, the shape of the object 100 can be properly grasped, and the shape of the object 100 can be properly measured with a simpler procedure.

Note that the shape measurement system 12 stores the imaging conditions for each of a plurality of combinations of the material and surface properties of the object 100 as condition information 42 in advance, and other configurations may be changed as appropriate as long as the imaging conditions for the object 100 are specified based on the condition information 42. For example, in the above description, in order to specify the material and surface properties of the object 100, the processing drawing 64 and processing program 66 are used preferentially, then the preliminary image 62 is used, and finally, the operator is asked to input information. However, the procedure for specifying the material and surface properties may be changed as appropriate. Therefore, for example, the operator may always be asked to input information. In another embodiment, the material and surface properties may be specified always using the preliminary image 62 without using the processing drawing 64 and processing program 66.

In the above description, depending on the material and surface properties of the object 100, the HDR image 72 is generated and treated as the measurement image 60, but if the shape of the object 100 can be grasped, the HDR image 72 does not need to be generated at all. In the above description, the measurement controller 30 and the numerical control device 20 are separate computers independent of each other, but the measurement controller 30 may be incorporated into the numerical control device 20. In other words, the numerical control device 20 may function as the measurement controller 30. The number of each component may also be changed as appropriate, and for example, the shape measurement system 12 may have multiple measurement controllers 30 and multiple imaging units 50. One machine tool system 10 may have multiple shape measurement systems 12. In the above description, the projector 54 is used as the light source 53, but the light source 53 may be another device as long as it can irradiate the object 100 with light suitable for imaging, i.e., imaging light. For example, the light source 53 may be a laser light source that emits a pattern light such as a line light or a point cloud light as the light for imaging. Also, in the above description, the shape measurement system 12 is incorporated into the machine tool system 10 having the machine tool 14. However, the shape measurement system 12 may be incorporated into a system unrelated to the machine tool 14, or may be provided independently without being incorporated into another system.

10 Machine tool system, 12 Shape measurement system, 14 Machine tool, 20 Numerical control device, 22 Operation panel, 30 Measurement controller, 32 Processor, 34 Memory, 36 Communication I/F, 38 UI device, 40 Storage device, 42 Condition information, 44 Reference data, 45 Learning model, 50 Imaging unit, 52 Camera, 53 Light source, 54 Projector, 56 Imaging conditions, 60 Measurement image, 62 Preliminary image, 64 Processing drawing, 66 Processing program, 69 Three-dimensional data, 70, 70f, 70s Basic image, 72 HDR image, 74 Point cloud data, 80a Material correspondence table, 80b Surface property correspondence table, 100 Object, 102 Protrusion, 104 Edge.

Claims (9)

An imaging unit having one or more cameras for imaging an object;
a storage device that stores, as condition information, imaging conditions for measurement imaging of the object for measuring the shape of the object for each of a plurality of combinations of the material and surface property of the object;
A measurement controller that controls the driving of the imaging unit;
The measurement controller includes:
identifying a material and a surface property of the object, identifying the imaging conditions corresponding to the identified material and surface property of the object based on the condition information, causing the imaging unit to perform the measurement imaging under the identified imaging conditions, and measuring the shape of the object based on the obtained measurement image.
A three-dimensional shape measuring system comprising: The three-dimensional shape measurement system according to claim 1 ,
The three-dimensional shape measurement system is characterized in that the measurement controller causes the imaging unit to image the object prior to the measurement imaging, and identifies at least one of the material and surface properties of the object based on the preliminary image obtained thereby. 3. The three-dimensional shape measuring system according to claim 2,
The storage device further stores a learning model that receives the preliminary image as an input and outputs at least one of the material and the surface texture of the object,
The measurement controller identifies at least one of a material and a surface property of the object based on the learning model.
A three-dimensional shape measuring system comprising: 4. The three-dimensional shape measurement system according to claim 1,
The imaging unit is provided in a machine tool and captures an image of a workpiece being machined by the machine tool as the target object;
The measurement controller identifies at least one of a material and a surface property of the object based on at least one of a machining program for the workpiece and a machining drawing for the workpiece.
A three-dimensional shape measuring system comprising: The three-dimensional shape measuring system according to claim 4,
the measurement controller acquires information indicating at least one of the machining program and the machining drawing from a numerical control device of the machine tool;
A three-dimensional shape measuring system comprising: 6. The three-dimensional shape measurement system according to claim 1,
2. A three-dimensional shape measuring system, wherein at least one of the material and the surface property of the object is specified by an operator. 7. The three-dimensional shape measurement system according to claim 1,
The imaging unit further includes one or more light sources that irradiate the object with imaging light,
the measurement controller executes basic imaging of the object one or more times to obtain one of the measurement images;
The imaging conditions include at least one of the number of times the basic imaging is performed to obtain one of the measurement images, the shutter speed of the camera in each of the basic imaging performed one or more times, the gain of the camera in each of the basic imaging performed one or more times, and the luminance distribution of the imaging light in each of the basic imaging performed one or more times.
A three-dimensional shape measuring system comprising: The three-dimensional shape measurement system according to any one of claims 1 to 7,
the measurement controller generates point cloud data of the object based on the measurement images, and generates three-dimensional data of the object from the point cloud data.
A three-dimensional shape measuring system comprising: The three-dimensional shape measuring system according to claim 8 ;
A machine tool that performs machining on the object;
Equipped with
The machine tool performs at least one of the following, based on the three-dimensional data generated by the measurement controller: checking for interference between a tool and the object; generating a path for the tool; determining machining accuracy; determining whether the shape of the object coincides with a reference shape; and determining whether the object is located at a predetermined position.
A machine tool system comprising: