JP7548870B2 - Cylinder inner surface imaging device - Google Patents

Description

The present invention relates to a device for imaging the inner surface of a cylinder.

A surface inspection system has been proposed that can improve the detectability and visibility of defects in recessed positions by performing a geometric transformation on a wide-angle image acquired by inserting an endoscope into a cylindrical space formed in a workpiece, creating an unfolded image of the inner wall side with the depth direction and circumferential direction as coordinate axes, and removing trends by sequentially performing a first smoothing process along the depth direction and a second smoothing process along the circumferential direction on this unfolded image (Patent Document 1).

Patent Document 1 discloses a technology to eliminate the tendency for the amount of auxiliary light to be maximum near the tip and to rapidly attenuate as it moves away in the depth direction, due to the diffusion phenomenon of light when it is emitted from the tip of the endoscope to the outside.

JP 2014-96020 (Abstract, 0043, Figures 1 and 3)

However, when an endoscope is inserted deep into a bore and imaging is performed, a phenomenon occurs in which the image becomes brighter the further down the bore is, due to the illumination being reflected from the bottom surface, which is the opposite of the attenuation of light in the depth direction as pointed out in Patent Document 1. Also, even if there are irregularities on the side surface or the material or surface treatment is different, the image will be locally brighter. For this reason, if images are acquired by sequentially capturing images while the endoscope is inserted into the bore, and then images near the insertion depth are cut out and combined to obtain an image of the entire inner surface, uneven brightness will occur, which is the opposite of what is pointed out in Patent Document 1.

It is also possible to adjust the brightness of the captured image, but this is not possible if the brightness exceeds the upper or lower limit (saturates) and brightness information is lost in that area.

The present invention aims to reduce brightness unevenness in a combined image obtained by sequentially capturing images of the inner surface of a cylinder in the depth direction.

The cylindrical inner surface imaging device of the present invention, which has been made to achieve this object, comprises an endoscope that can be inserted into a cylindrical portion formed in a workpiece, an illumination means for irradiating light into the cylindrical portion through the endoscope, an imaging means for acquiring a wide-angle image of the inner surface of the cylindrical portion through the endoscope, an industrial robot having at least a lifting function so as to insert the endoscope into the cylindrical portion while controlling the insertion depth, and an information processing means for generating an unfolded image of the inner surface of the cylindrical portion based on the wide-angle image captured by the imaging means, and further comprises the following configuration.
(1A) The imaging means is configured as a means for acquiring a wide-angle image of the range illuminated by the lighting means when the endoscope is inserted to a predetermined depth by the industrial robot, and the information processing means is configured as a means for generating the inner surface unfolded image by combining, in the depth direction of the cylindrical portion, multiple unfolded images obtained by cutting out a predetermined range of the peripheral portion for each of multiple wide-angle images acquired at multiple points in the depth direction of the cylindrical portion and unfolding them in polar coordinates.
(1B) An imaging schedule storage means is provided for storing an imaging schedule in which, among the imaging conditions for obtaining a wide-angle image by the imaging means, the light emission conditions in the illumination means or the exposure conditions in the imaging means are set as individual conditions that are changed according to the insertion depth of the endoscope so as to reduce brightness unevenness in the internal surface unfolded image, and the imaging schedule is stored in association with work information that identifies the cylindrical portion to be imaged.
(1C) An operation control means is provided which reads out an imaging schedule for the cylindrical part to be imaged from the imaging schedule storage means based on the work information, and controls the operation of the industrial robot, the lighting means, and the imaging means in accordance with the imaging schedule.

According to the cylindrical inner surface imaging device of the present invention, when work information is given that specifies a work having a cylindrical portion to be imaged, the operation control means reads out the corresponding imaging schedule from the imaging schedule storage means, and controls the operation of the industrial robot, the lighting means, and the imaging means according to this imaging schedule. As a result, when the industrial robot inserts the endoscope to a predetermined depth into the cylindrical portion of the work, an operation of having the imaging means acquire wide-angle images of the range illuminated by the lighting means is sequentially performed in the depth direction of the cylindrical portion. During this, the information processing means generates an inner surface developed image by combining multiple developed images obtained by cutting out a predetermined range of the peripheral portion from each of the multiple wide-angle images obtained at multiple locations in the depth direction of the cylindrical portion and developing them in polar coordinates in the depth direction of the cylindrical portion. For the same type of work, inner surface developed images of the cylindrical portion of each work are generated according to the same imaging schedule. When the work to be imaged is changed, the work information of the changed work is given, and the above-mentioned operation is performed by the operation control means, and the information processing means generates an inner surface developed image of the cylindrical portion of the work by combining multiple developed images obtained by cutting out a predetermined range of the peripheral portion of the wide-angle images obtained at multiple locations in the depth direction and developing them in polar coordinates in the depth direction of the cylindrical portion. Here, the imaging schedule for each workpiece is an imaging schedule in which the imaging conditions for obtaining a wide-angle image by the imaging means from the range illuminated by the lighting means are set as individual conditions according to the insertion depth of the endoscope, so that different imaging conditions can be set for the top and bottom of the cylindrical portion, preventing uneven brightness in the internal development image due to the influence of reflected light near the bottom.

Here, it is preferable that the cylindrical inner surface imaging device of the present invention further comprises the following configuration.
(2A) A teaching means is provided for teaching the industrial robot an operation for acquiring the wide-angle images at a plurality of locations in the depth direction of the cylindrical portion to be imaged.
(2B) An imaging schedule adding means is provided for determining the imaging conditions while the teaching means is teaching the operation of the industrial robot, and storing the imaging conditions in the imaging schedule storage means as an imaging schedule for the workpiece that was the subject of the teaching, together with workpiece information that identifies the workpiece.

According to a cylindrical inner surface imaging device having such a configuration, when the teaching means teaches the industrial robot the operation of inserting an endoscope into the cylindrical portion of each workpiece, the imaging schedule adding means determines imaging conditions for reducing unevenness in brightness in the inner surface developed image generated from wide-angle images acquired at multiple points in the depth direction, and stores the imaging schedule for the workpiece that was the subject of the teaching in the imaging schedule storage means together with workpiece information that identifies the workpiece. This makes it easy to set optimal imaging conditions for each imaging target.

The cylindrical inner surface imaging device of the present invention may further include the following configuration.
(3) The illumination means is configured as a means for executing strobe emission, and the imaging schedule is set so that the imaging conditions are set so as to reduce brightness unevenness in the internally expanded image, at least by increasing or decreasing the emission period of strobe emission for each location where the wide-angle image is acquired.

As imaging conditions for reducing brightness unevenness in the internal development image, "(a) a configuration for increasing or decreasing the illuminance in the illumination means" and "(b) a configuration for increasing or decreasing the exposure conditions and gain brightness in the imaging means" can also be applied according to the insertion depth of the endoscope. However, while (a) "increasing or decreasing the illuminance" can cause new problems such as increased costs and complicated control of the illumination means, and (b) "changing the conditions on the imaging means" can cause new problems such as increased costs and complicated control of the imaging means, as described above, "(c) increasing or decreasing the light emission period of the strobe light" can be achieved by "lengthening or shortening the power supply time", and it is possible to reduce brightness unevenness without causing new problems such as increased costs and complicated control of the illumination means and imaging means.

According to the present invention, it is possible to reduce uneven brightness in a combined image obtained by sequentially capturing images of the inner surface of a cylinder in the depth direction.

1A and 1B are perspective views showing the cylindrical inner surface imaging device of Example 1 when attached to an articulated robot for use, and FIG. 1C is an enlarged perspective view of the main parts showing the relationship between irradiated light and reflected light during imaging. 1A shows a cylinder inner surface imaging device of Example 1, in which (A) is a block diagram of a control system, (B) is a flowchart of a teaching routine, and (C) is a flowchart of an imaging condition determination routine. 1 shows a cylindrical inner surface imaging device according to a first embodiment, in which (A) is a flowchart of an imaging routine, and (B) is a flowchart of an unfolded image generating routine. 3A to 3C are schematic diagrams illustrating the action and effect of the cylindrical inner surface imaging device of the first embodiment.

Below, the form for implementing the present invention will be explained based on the examples.

The cylindrical inner surface imaging device 10 of the first embodiment is attached to an articulated robot 1 as shown in FIG. 1, and is adapted to capture an image of the inner surface of the cylindrical portion Wh by moving the endoscope 11 so that its axis is aligned with the center of the cylindrical portion Wh of the workpiece W, while capturing an image with the imaging device 13 at the timing when the lighting device 12 emits a strobe light. At this time, as shown in FIG. 1(C), when the endoscope 11 is inserted deep inside the cylindrical portion Wh, there is a concern that the imaging range may become too bright due to reflected light from near the bottom surface. The cylindrical inner surface imaging device 10 of the present embodiment is adapted to solve this problem.

The control system of the cylindrical inner surface imaging device 10 of the embodiment is configured around a calculation device 100, as shown in FIG. 2(A). The calculation device 100 is configured as a personal computer equipped with input devices such as a monitor 110, a storage device 120, a keyboard 131, a mouse 132, and a cursor key 133. The calculation device 100 transmits control commands to the articulated robot 1, the lighting device 12, and the imaging device 13 to perform imaging of the cylindrical part.

A teaching program (Figure 2(B)), an imaging condition determination program (Figure 2(C)), an imaging program (Figure 3(A)), and an unfolded image generation program (Figure 3(B)) are installed in the memory device 120 of the calculation device 100.

When the teaching program is started, as shown in FIG. 2(B), the calculation device 100 waits for the input of work information (S10). The operator inputs the work information by operating input devices such as the keyboard 131, mouse 132, and cursor keys 133.

The work information includes a work ID that identifies the work W to be imaged on the cylindrical inner surface, information about the work itself such as the material, surface treatment, and work dimensions of the work, information about the cylindrical part that identifies the position, hole diameter, depth, and other dimensions of the cylindrical part formed in the work, and information about the generation of the expanded image, such as the imaging pitch in the depth direction and the brightness of the expanded image obtained by polar coordinate expansion of a specified range of the peripheral part of the captured image. Here, the work ID may be configured to automatically assign a unique ID. Also, the imaging start position, end position, and movement pitch may be directly input. In addition, the illumination intensity, camera gain, brightness, gamma correction value, and the like may be input as reference imaging conditions. At this time, the calculation device 100 displays an input screen for work information on the monitor 110 to prompt the operator to input the work information.

When the work information is input (S10: YES), the calculation device 100 stores the work information in the storage device 120 (S20). When storage of the work information is thus completed, the calculation device 100 calculates an imaging schedule based on the work information (S30).

This imaging schedule aligns the center of the endoscope 11 with the center of the cylindrical part of the workpiece, positions the tip of the endoscope 11 at the height where imaging starts, and then has the articulated robot 1 sequentially lower the endoscope 11 in a straight line at a predetermined pitch from the top of the cylindrical part to an imaging end position near the bottom of the cylindrical part, stopping the operation, and when the descent stops, has the lighting device 12 emit a strobe light for a predetermined period of time and has the imaging device 13 capture a wide-angle image at that time. If the workpiece information stored by S10 and S20 is different, the imaging schedule is also calculated as being different. Note that, in order to calculate this imaging schedule, information is stored in advance in the storage device 120 that associates a reference illuminance and a reference emission period, with the parameters being the material and surface treatment of the workpiece and the hole diameter of the cylindrical part.

Next, the arithmetic device 100 waits for the workpiece to be taught this time to be set (S40). At this time, the arithmetic device 100 displays a message on the monitor 110 prompting the operator to set the workpiece. Then, when the operator inputs that the workpiece has been set (S40: YES), the arithmetic device 100 outputs a command to the articulated robot 1 to align the center of the endoscope 11 with the center of the cylindrical part of the workpiece and position the tip of the endoscope 11 at the height that will be the imaging start position according to the imaging schedule (S50).

After moving the endoscope 11 to the imaging start position in this manner, the calculation device 100 executes an imaging condition determination process (S100). When this imaging condition determination process starts, as shown in FIG. 2(C), the calculation device 100 sets the imaging point n to "1" (S110), outputs a command to the articulated robot 1 to lower the endoscope 11 by a height Hn according to the imaging schedule calculated in S30, and when an operation completion signal is received from the articulated robot 1, causes the illumination device 12 to emit a strobe light based on the light emission condition Ln and causes the imaging device 13 to latch the wide-angle image PHTn at that time (S120).

Next, a predetermined range of the periphery of the latched wide-angle image PHTn is cut out, and an expanded image EXPIMGn is generated by polar coordinate expansion and stored in the storage device 120 (S130). After that, it is determined based on the imaging schedule whether the maximum depth to be imaged has been reached in the cylindrical portion Wh (S140), and if not, the imaging point n is incremented (S150) and the process returns to S120.

In this way, the processes of S120 to S130 are repeated until the maximum depth to be imaged is reached for the cylindrical portion (S140; YES), the expanded images EXPIMGn stored in the memory device 120 are read out, and the combined image CMBIMG, which is combined in the depth direction, is displayed on the monitor 110 (S150), and the system waits for operator input regarding the presence or absence of any imaging points that require adjustment of the light emission conditions Ln (S160).

When an input is made that there is an imaging point that requires adjustment of the light emission condition Ln (S160: YES), a message prompting the user to specify the imaging point that requires adjustment is displayed on the monitor 110, and the system waits for the user to specify the adjustment point m (S170).

When an imaging point m that needs to be adjusted is specified (S170: YES), the imaging schedule is updated so that the light emission period as the light emission condition Lm for that imaging point m is increased or decreased by the unit time Δt (S180), and the process returns to S110.

Here, whether the unit time Δt is to be increased or decreased may be input by the operator, or it may be preset to be increased or decreased.

This update of the imaging schedule is repeated until the operator makes an input indicating that there are no more imaging points that require adjustment of the light emission condition Ln. Then, when the operator makes an input indicating that there are no more imaging points that require adjustment of the light emission condition Ln (S160: NO), the latest imaging schedule is stored in the storage device 110 together with the work information (S190), and the process returns to the main flow of the teaching routine.

Once the imaging schedule for one workpiece has been determined in this manner, it is determined whether there is a next workpiece for which an imaging schedule should be determined (S200). The operator operates the input device to input whether there is a next workpiece. If there is a next workpiece (S200: YES), the process proceeds from S10. Once the imaging schedule for all workpieces to be imaged has been determined (S200: NO), the teaching routine ends.

Next, the imaging process of the cylindrical inner surface in this embodiment will be described with reference to the flowchart in FIG. 3. When the operator inputs work information (S310: YES), the computing device 100 reads out the imaging schedule stored in association with the work information from the storage device 110 (S320) and waits for the work to be imaged this time to be set (S330). When the operator inputs that the work has been set (S330: YES), the computing device 100 outputs a command to the articulated robot 1 to align the center of the endoscope 11 with the center of the cylindrical part of the work and position the tip of the endoscope 11 at the height where imaging will start (S340) according to the imaging schedule read out in S320.

Once the endoscope 11 has been moved to the imaging start position in this manner, the computing device 100 executes imaging processing (S400). When this imaging processing starts, as shown in FIG. 3(C), the computing device 100 sets the imaging point n to "1" (S310), outputs a command to the articulated robot 1 to lower the endoscope 11 by height Hn according to the imaging schedule read out in S320, and when an operation completion signal is received from the articulated robot 1, causes the lighting device 12 to emit a strobe light based on the light emission condition Ln and causes the imaging device 13 to latch the wide-angle image PHTn at that time (S420).

Next, a predetermined range of the periphery of the latched wide-angle image PHTn is cut out, and an expanded image EXPIMGn is generated by polar coordinate expansion and stored in the storage device 120 (S430). After that, it is determined based on the imaging schedule whether the maximum depth to be imaged has been reached in the cylindrical portion (S440), and if not, the imaging point n is incremented (S450) and the process returns to S420.

In this way, the processes of S420 to S430 are repeated until the maximum depth to be imaged is reached with respect to the cylindrical portion (S440; YES), the expanded images EXPIMGn stored in the memory device 120 are read out and a combined image CMBIMG is generated by combining them in the depth direction (S460), and the combined image CMBIMG thus generated is stored in the memory device 120, and the process returns to the main routine of the imaging process (S470).

When the process returns to the main routine of the imaging process, the computing device 100 determines whether or not there are any workpieces of the same type remaining to be imaged (S500). If there are any workpieces of the same type remaining to be imaged (S500: YES), the computing device 100 returns to the process of S330. On the other hand, if there are no workpieces of the same type remaining to be imaged (S500: NO), the computing device 100 determines whether or not to image a workpiece of a different type (S600). If an image of a workpiece of a different type is to be imaged (S600: YES), the computing device 100 returns to S310 and requests input of workpiece information for that workpiece. If new workpiece information is input (S310: YES), the computing device 100 reads out the imaging schedule stored in correspondence with the newly input workpiece information (S320) and executes the processes from S330 onward.

The action and effect of this embodiment will be explained with reference to the schematic diagram in Figure 4. As shown, when imaging is performed under the same lighting conditions at all imaging points, a wide-angle image PHT1 that is dark at the top of the hole is captured, whereas a wide-angle image PHT4 that is bright overall is captured near the bottom of the hole due to the influence of reflected light from the bottom surface. Expansion images EXPIMG1 to EXPIMG5 obtained by cutting out a specified range of the peripheral portion based on wide-angle images PHT1 etc. captured under the same lighting conditions have different brightness in the depth direction of the cylindrical portion wh, and the combined image CMBIMG generated from these has uneven brightness in the depth direction. In contrast, the expanded images EXPIMG1c to EXPIMG5c obtained by cutting out a specified peripheral range based on the wide-angle image PHT1c, etc., captured based on the imaging schedule obtained by determining the optimal light emission conditions for each imaging point determined by the teaching process of this embodiment, all have approximately the same brightness, and the combined image CMBIMGc generated from these has no uneven brightness in the depth direction.

By subjecting the combined image CMBIMGc obtained in this way to a judgment process using image processing techniques, etc., highly reliable inspection results can be obtained.

The above describes an embodiment of the present invention, but the present invention is not limited to this and can be implemented in various ways without departing from the spirit of the invention.

For example, the necessity of adjusting the light emission conditions in the imaging condition determination process may be determined by the image processing program. In the embodiment, a case where an endoscope insertion operation is taught using an articulated robot as an example of an industrial robot has been described, but an imaging schedule in which optimized imaging conditions are set when teaching an endoscope insertion operation using a scalar robot or an orthogonal robot instead of an articulated robot may be added. In this case, when an orthogonal robot having only a linear axis in the lifting direction is used as the industrial robot, a jig for each workpiece may be used to align the axis of the cylindrical part and the endoscope. In addition, the lighting device is not limited to strobe lighting, but may be a type that performs continuous lighting, and imaging conditions that change the exposure conditions on the imaging device side depending on the insertion depth of the endoscope may be set as an imaging schedule.

The present invention can be used to inspect the inner surfaces of cylindrical parts of various workpieces.

1: Articulated robot, 10: Cylinder inner surface imaging device, 11: Endoscope, 12: Illumination device, 13: Imaging device, CMBIMG, CMBIMGc: Combined image, EXPIMG1-EXPIMG5, EXPIMG1c-EXPIMG5c: Expanded image, PHT1-PHT4, PHT1c-PHT4c: Wide-angle image, W: Workpiece, Wh: Cylinder part.

Claims (3)

A cylindrical inner surface imaging device comprising an endoscope that can be inserted into a cylindrical portion formed in a workpiece, an illumination means for irradiating light into the cylindrical portion through the endoscope, an imaging means for acquiring a wide-angle image of the inner surface of the cylindrical portion through the endoscope, an industrial robot having at least a lifting function so as to insert the endoscope into the cylindrical portion while controlling the insertion depth, and an information processing means for generating an unfolded image of the inner surface of the cylindrical portion based on the wide-angle image captured by the imaging means, and further comprising the following configuration.
(1A) The imaging means is configured as a means for acquiring a wide-angle image of the range illuminated by the lighting means when the endoscope is inserted to a predetermined depth by the industrial robot, and the information processing means is configured as a means for generating the inner surface unfolded image by combining, in the depth direction of the cylindrical portion, multiple unfolded images obtained by cutting out a predetermined range of the peripheral portion for each of multiple wide-angle images acquired at multiple points in the depth direction of the cylindrical portion and unfolding them in polar coordinates.
(1B) An imaging schedule storage means is provided for storing an imaging schedule in which, among the imaging conditions for obtaining a wide-angle image by the imaging means, the light emission conditions in the illumination means or the exposure conditions in the imaging means are set as individual conditions that are changed according to the insertion depth of the endoscope so as to reduce brightness unevenness in the internal surface unfolded image, and the imaging schedule is stored in association with work information that identifies the cylindrical portion to be imaged.
(1C) An operation control means is provided which reads out an imaging schedule for the cylindrical part to be imaged from the imaging schedule storage means based on the work information, and controls the operation of the industrial robot, the lighting means, and the imaging means in accordance with the imaging schedule. 2. The cylindrical inner surface imaging device according to claim 1, further comprising the following configuration:
(2A) A teaching means is provided for teaching the industrial robot an operation for acquiring the wide-angle images at a plurality of locations in the depth direction of the cylindrical portion to be imaged.
(2B) The present invention is provided with an imaging schedule adding means for determining the imaging conditions while teaching the operation of the industrial robot by the teaching means, and storing the imaging conditions in the imaging schedule storage means together with work information identifying the work as an imaging schedule for the work that was the subject of the teaching. 3. The cylindrical inner surface imaging device according to claim 1, further comprising the following configuration:
(3) The illumination means is configured as a means for executing strobe emission, and the imaging schedule is set so that the imaging conditions are set so as to reduce brightness unevenness in the internally expanded image, at least by increasing or decreasing the emission period of strobe emission for each location where the wide-angle image is acquired.