JP7510459B2 - Tunnel face evaluation system and tunnel face evaluation method - Google Patents

Description

The present invention relates to a tunnel face evaluation system and a tunnel face evaluation method, and in particular to a tunnel face evaluation system and a tunnel face evaluation method using the same, which acquire images of the face stored in cloud storage and generate a deformed base image for sketching by color-coding the face image according to the overall appearance of the color tone using an image conversion processing means using deep learning.

At mountain tunnel sites, face observation is a routine task in which the rock mass that appears at the tunnel face is observed and recorded in order to select the support structure (support pattern) and auxiliary construction methods.
Face observation involves approaching the tunnel face immediately after excavation, and the act of recording the evaluation items determined on the spot is a task that involves the risk of disaster or accidents, such as contact with heavy machinery or skin falls.

The task of face observation requires the preparation of a document called a face observation record, which basically consists of a sketch, which is a diagram of the face observed and illustrated, with comments added where necessary, and an evaluation category, which is a table in which the face is broken down into evaluation areas of the top, left shoulder, and right shoulder, and evaluation points are calculated for each evaluation area and for items such as weathering and crack conditions, making the preparation of the face observation record time-consuming. As the face changes as the excavation of the tunnel progresses, face observation records must be prepared regularly, and there is a problem of a large burden on the work in terms of frequently preparing the observation record documents, and there is a demand for labor-saving measures.
To address these issues, research is also underway into technologies to assist in observing the tunnel face.

Patent Document 1 discloses a face evaluation support system that includes a learning result storage unit that stores learning results generated by machine learning using a learning divided image generated by dividing an image of a face photographed during underground observation by a reference number of pixels as an input layer and an evaluation category for the observation item of the face as an output layer, and a control unit connected to the input unit and output unit, in which the control unit divides the evaluation area of the face image to be evaluated acquired from the input unit by a reference number of pixels, applies the learning result to the divided divided image of the evaluation object to obtain an individual evaluation category, predicts the evaluation category of the evaluation area based on the individual evaluation category, and outputs it to the output unit.

The invention of Patent Document 1 mainly uses deep learning technology to directly calculate an evaluation score from an image. Although past research results have confirmed that the method of directly calculating an evaluation score from image data using deep learning processing can evaluate with a certain degree of accuracy, there is a problem in that it is difficult to explain points such as "what contributes to high accuracy" and "whether deep learning is really performing appropriate processing."
In addition, in the invention of Patent Document 1, the evaluation category is output by the evaluation support device, which can contribute to labor savings, but it does not support the creation of sketches of the face, etc., and there is still room for labor savings.

Therefore, it is desirable to provide a tunnel face evaluation system that reduces the labor required for face observation by generating original drawings for making sketches based on images of the face taken by the worker, generating evaluation points based on the generated original drawings, and generating manuscripts that serve as the basis for face observation records based on the original drawings for making sketches and the generated evaluation points.

JP 2019-023392 A

The present invention has been made in consideration of the problems with the conventional tunnel face evaluation system described above, and the object of the present invention is to provide a tunnel face evaluation system and a tunnel face evaluation method using the same, which acquires an image of the face stored in cloud storage and uses an image conversion processing means using deep learning to generate a deformed base image for sketching that is color-coded according to the overall appearance of the image of the face.

The tunnel face evaluation system according to the present invention, which has been made to achieve the above-mentioned object, is a tunnel face evaluation system that includes an evaluation server that is connected to a mobile terminal of a worker via a cloud storage network, and the evaluation server includes a sketch base image generation unit that acquires an image of the face acquired by the mobile terminal and stored in the cloud storage, and generates a sketch base image that is deformed by color-coding the image of the face according to the color tone of the appearance from a global perspective using an image conversion processing means using deep learning, a face evaluation unit that generates an evaluation score for the face based on the generated sketch base image, and a face observation record manuscript generation unit that generates a face observation record manuscript based on the sketch base image and the evaluation score and stores it in the cloud storage.

It is preferable that the evaluation server further includes a face location detection unit that acquires an image of the face and detects the face location by an object detection means using deep learning, or acquires an image of the sketch base and detects the face location from a color analysis of the sketch base image, and a face location decomposition processing unit that decomposes the face location into the top, left shoulder, and right shoulder evaluation areas.

The tunnel face evaluation method according to the present invention, which has been made to achieve the above object, is a tunnel face evaluation method using a tunnel face evaluation system equipped with an evaluation server and a mobile terminal connected by a network via cloud storage, and is characterized by having a step of acquiring an image of the tunnel face with the mobile terminal of an operator and storing it in the cloud storage, a step of the evaluation server acquiring the image of the tunnel face stored in the cloud storage and generating a sketch base image from the image of the tunnel face, which is color-coded according to the color tone of the overall appearance by an image conversion processing means using deep learning, a step of generating an evaluation score for the tunnel face based on the generated sketch base image, a step of generating a tunnel face observation record manuscript based on the sketch base image and the evaluation score, and a step of storing the tunnel face observation record manuscript in the cloud storage.

It is preferable to further include a step of acquiring the face observation record manuscript stored in the cloud storage by the mobile terminal, adding supplements or corrections to the face observation record manuscript, and storing it in the cloud storage as a completed face observation record.

The step of generating an evaluation score for the face based on the sketch base image preferably includes a step of detecting the face location from the face image or the sketch base image, a step of decomposing the detected face location into evaluation areas of the top, left shoulder, and right shoulder, and a step of calculating a weathering deterioration evaluation score based on the area of a color preset for each of the decomposed evaluation areas.

The step of detecting the face location preferably includes the steps of detecting the face location from the image of the face using an object detection means using deep learning, judging whether the detected face location is appropriate, and adopting the detected face location if it is judged that the detected face location is appropriate, and detecting the face location from the sketch base image and adopting the face location detected from the sketch base image if it is judged that the detected face location is inappropriate.

According to the tunnel face evaluation system of the present invention, a worker simply uses a mobile device to acquire an image of the face and saves the acquired image of the face in cloud storage. The evaluation server detects that a new image of the face has been saved to cloud storage, and generates a sketch base image that serves as the basis for a sketch based on the saved image of the face. This allows a worker to easily create a sketch by simply adding the necessary comments to the sketch base image.

In addition, with the tunnel face evaluation system of the present invention, the face is evaluated by first generating a sketch base image that is deformed and color-coded according to the overall appearance of the color tone, and then the evaluation result is calculated based on the generated sketch base image. This makes it easy to understand how the evaluation result was calculated, and the system is less likely to raise questions about whether deep learning is performing appropriate processing.

Furthermore, according to the tunnel face evaluation system of the present invention, the face location is detected by the evaluation server and broken down into evaluation areas for the top, left shoulder, and right shoulder, and then output, eliminating the need for workers to cut out images. In addition, the sketch base image generated by the evaluation server and the evaluation results calculated based on the sketch base image are output as a face observation record manuscript that can be compiled into a face observation record with just a little modification by the worker, making it possible to significantly reduce the effort required to create a face observation record.

1 is a block diagram showing an outline of the configuration of a tunnel face evaluation system according to an embodiment of the present invention. FIG. 2 is a diagram showing a schematic flow of processing in a tunnel face evaluation system according to an embodiment of the present invention. 13A and 13B are diagrams showing examples of an acquired face image and a deformed sketch base image generated from the face image. 13 is a diagram showing an example of the results of detecting the face location from an acquired image of the face directly or via a sketch base image. FIG. This figure shows an example of the results of generating weathering and deterioration evaluation points for a face from a sketch base image decomposed into evaluation areas. FIG. 2 is a diagram showing an example of a working face observation record manuscript according to an embodiment of the present invention. 1 is a flowchart for explaining a tunnel face evaluation method according to an embodiment of the present invention. 1 is a flowchart for explaining a tunnel face evaluation point generation method for calculating weathering deterioration evaluation points in a tunnel face evaluation method according to an embodiment of the present invention.

Next, a specific example of an embodiment for carrying out the tunnel face evaluation system according to the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of a tunnel face evaluation system according to an embodiment of the present invention.
Referring to FIG. 1 , a tunnel face evaluation system 1 according to an embodiment of the present invention includes an evaluation server 10 and a mobile terminal 20 of a worker, which are mutually connected via a network via a cloud storage 5 .

The tunnel face evaluation system 1 according to an embodiment of the present invention is a system in which an evaluation server 10 that evaluates the state of a tunnel face acquires an image of the tunnel face acquired by a mobile terminal 20 and stored in cloud storage 5, and generates a sketch base image that is deformed by color-coding the overall appearance of the color tone from the image of the face using an image conversion processing means that uses deep learning, generates an evaluation score for the face based on the generated sketch base image, and further generates a face observation record draft based on the sketch base image and the face evaluation score and stores it in cloud storage 5, thereby enabling workers to significantly reduce the effort required to create a face observation record of the tunnel face.

In order to perform the above functions, the evaluation server 10 is equipped with an input/output unit 12, a memory unit 13, a display unit 14, a sketch base image generation unit 15, a face location detection unit 16, a face location decomposition processing unit 17, a face evaluation unit 18, a face observation record manuscript generation unit 19, and a control unit 11 that controls these elements.

The sketch base image generating unit 15 is equipped with an image conversion processing means using deep learning, and acquires an image of the face acquired by the mobile terminal 20 and stored in the cloud storage 5, and generates a sketch base image from the face image by the image conversion processing means, which is color-coded according to the color tone of the appearance from a global perspective. The sketch base image is an image that will ultimately be the basis for the sketch to be recorded in the face observation record, and is color-coded globally without being bound by the detailed structure, so that it is a base image that clearly expresses the color tone composition, including the weathering of the entire face, for anyone to see.

The sketch base image is also the basis for evaluating the weathering and deterioration of the face. For this reason, the sketch base image can be said to be a generated drawing at an intermediate stage when evaluating the weathering and deterioration of the face from the base image of the face, and the generation of the sketch base image also plays a role in making it easier to understand how the image conversion processing means using deep learning made the judgment to evaluate the face.

The sketch base image is part of the data output from the evaluation server 10 to support the creation of face observation records, and the worker can complete the sketch by acquiring the sketch base image and adding information about cracks, springs, etc. This saves labor compared to cutting out the necessary parts from the face image to create a sketch.

The image conversion processing means provided in the sketch base image generating unit 15 is an application of an image conversion type Generative Adversarial Network (GAN) as deep learning. The Generative Adversarial Network includes two networks, a generation network that generates image data and a discrimination network that judges the authenticity of the generated image data. The generation network changes parameters so that the generated image data is judged as true as possible, and the discrimination network changes parameters so that the generated image data is judged as false as possible, thereby improving the accuracy of the generated image data.
The image conversion processing means stores the generative adversarial network, which has thus come to generate image data that satisfies specified specifications by repeatedly learning to generate and judge image data, in a memory unit within the image conversion processing means as a deep learning model.

The deep learning model provided in the image conversion processing means reads in many images of the face, colors them according to rock type and weathering, outputs image data, and repeatedly determines the authenticity of the output image data, improving the accuracy of the deep learning model. Image data generated by the deep learning model with improved accuracy based on the acquired face image is the base image for sketching. The image conversion processing means is configured to generate a deformed base image for sketching by having it learn to emphasize the apparent color tone rather than the fine distribution of rock types when coloring them.

In an embodiment, the sketch base image generating unit 15 is configured to generate a sketch base image, calculate the area ratio of each color in the sketch base image based on the use of different colors used to generate the sketch base image, and graph the calculated area ratios and output them together with the sketch base image.

The image of the face is acquired at a predetermined timing according to the state of the face, which changes as the tunnel excavation work progresses. Therefore, for example, it is repeatedly acquired every construction start date or at the time of specific progress. Therefore, in the embodiment, in order to reduce the burden on the worker or the operator of the evaluation server 10, a cloud storage 5 with file sharing and notification functions such as Dropbox (registered trademark) is used as the cloud storage 5, and the worker simply acquires the image of the face using the mobile terminal 20 and saves it in the cloud storage 5 with a specific file name, and the sketch base image generation unit 15 can detect the update of the image of the face via the input/output unit 12 and acquire the saved image of the face. The sketch base image generation unit 15 generates a sketch base image by an image conversion processing means using deep learning based on the acquired image of the face, and the completed sketch base image is handed over to the face observation record manuscript generation unit 19 as part of the data of the face observation record manuscript, which is the final output.

The face detection unit 16 acquires an image of the face and detects the face using an object detection means using deep learning, or acquires a sketch base image generated by the sketch base image generation unit 15 and detects the face from a color analysis of the sketch base image.
The object detection means repeatedly memorizes the characteristics of the upper half of the face to be evaluated using a large number of face images, and stores the results of learning which part of the face image has what characteristics the face location, in a memory unit provided in the object detection means.When a new face image is input, the means is configured to refer to the stored learning results to determine where the face location is and output the result.

Because there are various factors that cause the image of the face to change depending on the lighting and color conditions of the face when the image is acquired, it is expected that the detection of the face location may include errors and misjudgments. Therefore, the face location detection unit 16 judges whether the detected face location is appropriate. Specifically, for example, based on data on the face location of a tunnel under construction that has already been detected and approved by the work personnel, it judges whether the dimensional data of the detected face location, such as the width, height, or aspect ratio, falls within a predetermined error range of the approved dimensional data of the face location. In another embodiment, the curve of the outer periphery of the detected face location may be detected, and whether the detected face location is appropriate may be judged based on the curvature, the rise angle with the base, etc.

When the face location detection unit 16 determines that the detected face location is appropriate, it adopts the detected face location and transfers it to the face location decomposition processing unit 17. When it determines that the detected face location is not appropriate, the face location detection unit 16 acquires a sketch base image and detects the face location from a color tone analysis of the sketch base image. In one embodiment, the color tone analysis is based on whether the color tone is black or a color tone other than black, and detects the part with a color tone other than black as the face location. In this case, too, it may be determined whether the detected face location is appropriate or not based on the dimensional data of the approved face location as described above. In this way, when the face location detection unit 16 determines that the face location detected from the face image is not appropriate, it adopts the face location detected from the color tone analysis of the sketch base image and transfers it to the face location decomposition processing unit 17.

The face location decomposition processing unit 17 overlays the sketch base image acquired from the sketch base image generation unit 15 and the face location acquired from the face location detection unit 16, and decomposes the face location into the evaluation areas of the top, left shoulder, and right shoulder. Specifically, for example, the face location is divided into two equal parts in the height direction, with the upper side being the top, and the lower side is further divided into two equal parts left and right, with the left side being the left shoulder and the right side being the right shoulder.

The face evaluation unit 18 generates an evaluation score for the face based on the generated sketch base image. At this time, the face evaluation unit 18 generates an evaluation score for each evaluation area of the top, left shoulder, and right shoulder decomposed by the face part decomposition processing unit 17. The evaluation scores include at least a weathering evaluation score, and the weathering evaluation score is calculated based on the area of a preset color and how much area the preset color occupies. In one embodiment, the preset color is mainly brown, and in evaluation areas with a large brown area, an evaluation score of 4 indicating that weathering is progressing is selected according to the evaluation category, and in evaluation areas with no or few brown areas, 1 or 2 indicating that weathering is not progressing is selected.
When dark streak-like cracks are present within the face, the face evaluation unit 18 generates an evaluation score based on the frequency and condition of the cracks in accordance with the evaluation category, for example, by assigning a higher evaluation score to cracks that are closer together and by assigning a higher evaluation score to cracks that have a worse degree of opening.

The face observation record manuscript generation unit 19 generates a face observation record manuscript based on the sketch base image acquired from the sketch base image generation unit 15 and the evaluation points acquired from the face evaluation unit 18, and saves it in cloud storage 5. The face observation record manuscript is a pre-formatted manuscript, and is a base manuscript that is ultimately completed as a face observation record by entering necessary items and pasting necessary information. The face observation record manuscript generation unit 19 pastes the sketch base image onto the formatted manuscript and enters the evaluation points acquired from the face evaluation unit 18 in the corresponding entry fields, thereby generating a face observation record manuscript that incorporates the information obtained by the evaluation server 10, thereby reducing the burden on the worker creating the face observation record.

Since the face observation record manuscript is generated in a common format for one tunnel construction, the face observation record manuscript generation unit 19 is equipped with an input means for accepting the format of the face observation record manuscript in advance and a storage unit for storing the accepted format. The format of the face observation record is generally determined for each tunnel construction client. Therefore, in one embodiment, the face observation record manuscript generation unit 19 is configured to store the face observation record format determined for each client in association with the client, include the client's information in the initial registration data such as the tunnel name at the start of the tunnel construction and the tunnel construction specifications, and extract the format corresponding to the client and generate the face observation record manuscript when there is a face observation record format registered and stored based on the initial registration data. In another embodiment, each time a face observation record format is newly specified, the format is stored in the storage unit, and the format stored in the storage unit is provided in a menu format for selection during the initial registration of the tunnel construction.

The input/output unit 12 includes a means for communicating with the cloud storage 5 to obtain images of the tunnel face from the cloud storage 5 and to store manuscripts of the tunnel face observation records in the cloud storage 5. Since communication with the cloud storage 5 is performed via the Internet, the communication means may include a communication means compatible with the Internet. The input/output unit 12 also includes input means such as a keyboard and a mouse for inputting data such as the initial registration data for the tunnel construction work into the evaluation server 10.

The storage unit 13 stores various data handled by the evaluation server 10, programs for controlling the evaluation server 10 itself, and the like. In the above explanation, it has been described that the storage units, such as the storage unit within the image conversion processing means that stores the deep learning model, the storage unit provided in the object detection means that stores the learned results, and the storage unit that stores the format accepted by the face observation record manuscript generation unit 19, are provided separately, but in the embodiment, all of these storage units are configured as part of the storage unit 13. The storage unit 13 is realized by a hard disk device, a semiconductor memory device, or the like.

The display unit 14 displays various data such as a sketch base image and a face observation record manuscript, and also displays, if necessary, an application program for image processing and a program such as a control program for controlling the evaluation server 10. The display unit 14 is realized by a display device such as a liquid crystal display.
The control unit 11 controls each component of the information providing server 10 to perform the functions described above in accordance with a control program stored in the storage unit 13. The control unit 11 is realized by a control circuit including a semiconductor device such as a microcomputer.

The worker's mobile terminal 20 is used to acquire images of the face, input missing data after acquiring the draft face observation record stored in cloud storage 5, complete the face observation record, and store it in cloud storage 5.
The mobile terminal 20 includes a control unit 21 , an input/output unit 22 , a storage unit 23 , a display unit 24 , an image acquisition unit 25 , and an edited data input unit 26 .

The image acquisition unit 25 photographs the face to acquire the image of the face and stores it in the memory unit 23. For this reason, the image acquisition unit 25 is realized by a camera equipped in a smartphone or tablet terminal. It is sufficient to take an image of the face from the front without distortion, but it is desirable to take an image under the same conditions as much as possible each time. Therefore, in one embodiment, when an image of the face is acquired using a pre-implemented application, guidelines are displayed on the display screen of the mobile terminal 20 so that the position and size of the face are consistent.

The edited data input unit 26 is used to retrieve the face observation record manuscript stored in the cloud storage 5 and add supplements or corrections to the face observation record manuscript. The face observation record manuscript has fields for inputting comments and evaluation points, and fields other than those generated and filled in in advance by the evaluation server 10 remain blank, so the worker inputs the necessary information into the blank fields.

In addition, since a sketch base image is attached to the face observation record manuscript, the worker can edit the sketch base image by specifying an area and writing a comment, etc., to complete the sketch. If necessary, parts that require attention can be cut out, enlarged, etc., and pasted onto the face observation record manuscript separately from the sketch as supplementary material for the sketch. In order to perform such editing, the edit data input unit 26 is provided with a means for inputting text, as well as a means for inputting free figures, such as when specifying an area. It is also provided with a drawing editing means for processing drawings, such as cutting out, enlarging, and pasting.
Once edited and completed, the face observation record is stored in cloud storage 5.

The input/output unit 22 includes a communication means for transmitting and receiving data to and from the cloud storage 5 in order to obtain the draft face observation record stored in the cloud storage 5 and to store the completed face observation record in the cloud storage 5. The communication means may include any communication means compatible with the Internet.

The memory unit 23 stores data such as images of the face and face observation records, as well as application programs that run on the mobile terminal 20 and control programs that control the mobile terminal 20.
The display unit 24 displays data such as images of the face and draft face observation records.
The control unit 21 controls each component of the mobile terminal 20 to perform the functions described above in accordance with the control program stored in the storage unit 23 .
The mobile terminal 20 is realized by a smartphone or tablet terminal equipped with a camera, and the text and figure input means provided in the edit data input unit 26 is realized by a touch panel of the smartphone or tablet terminal.

FIG. 2 is a diagram showing a schematic flow of processing in a tunnel face evaluation system according to an embodiment of the present invention.
Referring to FIG. 2, data exchange between the mobile terminal 20 and the assessment server 10 via the cloud storage 5 is indicated by numbers (1) to (5), and processing within the assessment server 10 is indicated by (a) to (e).

First, an image of the face is acquired by the mobile device 20, and as indicated by the arrow (1), the acquired image is sent to and stored in cloud storage 5. In an embodiment of the present invention, cloud storage 5 is used that has file sharing and notification functions, so the acquired image is saved with a specified name, and the cloud storage 5 notifies the evaluation server 10 that a new image of the face has been saved. As a result, the evaluation server 10 detects that a new image of the face has been saved in cloud storage 5, and as indicated by the arrow (2), it acquires the newly saved image of the face from cloud storage 5.

The evaluation server 10 operates an image conversion processing means using deep learning in the sketch base image generation unit 15 as shown by the arrow (a), and generates a sketch base image and detects the face location based on the acquired face image. The detection of the face location includes judging whether the detected face location is appropriate or not, and if it is judged to be inappropriate, detecting the face location based on the generated sketch base image. The generated sketch base image is output to the face observation record manuscript generation unit 19 for use in the face observation record manuscript as shown by the arrow (b).

The generated sketch base image and the detected face location are then superimposed, and as shown by the arrow in (c), the face location is sent to a face location decomposition processing unit 17 to decompose the face location into the top, left shoulder, and right shoulder evaluation areas. As shown by the arrow in (d), the decomposed face locations are evaluated by a face evaluation unit 18, and an evaluation score is generated for each area. As shown by the arrow in (e), the generated evaluation scores are passed to a face observation record manuscript generation unit 19, and are compiled into a face observation record manuscript together with the previously output sketch base image.

The face observation record manuscript is output and stored in cloud storage 5, as indicated by the arrow (3). When the face observation record manuscript is saved in cloud storage 5, the notification function of cloud storage 5 notifies the mobile terminal 20 that a new face observation record manuscript has been saved. It is possible that multiple tunnel construction projects are carried out in parallel at the same time, and images of the face are sent separately from the mobile terminals 20 at each construction site. Therefore, the face observation record manuscript generation unit 19 saves the face observation record manuscript in such a way that it can be identified which construction it relates to, for example by using the name of the tunnel or an identification code assigned individually to each construction project in the file name. This limits the mobile terminals 20 that are notified when the face observation record manuscript is saved, and prevents notifications from being sent to mobile terminals 20 at other construction sites.

The worker at the mobile terminal 20 who received the notification retrieves the newly saved draft face observation record from cloud storage 5 (arrow (4)), adds the necessary information to the sketch background image in the draft face observation record to complete the sketch, and adds evaluation points, comments, etc. to complete the face observation record. The completed face observation record is sent to and saved in cloud storage 5, as indicated by arrow (5).

FIG. 3 shows an example of an acquired image of a working face and a deformed sketch base image generated from the image of the working face.
Referring to Fig. 3, three examples (a), (b), and (c) are shown, and the top diagram of each example is the original face image 30, and the bottom diagram of each example is the sketch base image 31 generated based on the face image 30. Although color tones cannot be expressed in Fig. 3, in a color display, the dark parts of the sketch base image 31 in (a) are gray, and the dark parts of the sketch base image 31 in (b) and (c) are brown. This color difference is based on the color tone of the original face image 30.
In the original faceplate image 30, it can be seen that there are scattered areas of different color tones in the details, but since the sketch base image 31 is represented by being color-coded and deformed according to the overall appearance of the color tone, the differences in the color tones in the details within the overall color tone are eliminated.

FIG. 4 is a diagram showing an example of the result of detecting the face location directly from the acquired image of the face or via a sketch base image.
4, the case where the face location 32-1 is detected directly from the original face image 30 shown in FIG. 4(a) (FIG. 4(c)) and the case where the sketch base image 31 shown in FIG. 4(b) is generated from the original face image 30 and then the face location 32-2 is detected based on the sketch base image 31 (FIG. 4(d)) are shown. The face location 32-1 in FIG. 4(c) is detected wider than the actual face location, whereas the face location 32-2 in FIG. 4(d) properly detects the actual face location. In the example of FIG. 4, the face location 32-1 in FIG. 4(c) is detected wider than the actual face location, so the aspect ratio of the face location 32-1 is different from the aspect ratio of the correct face location and is therefore determined to be inappropriate, and the face location 32-2 having the same aspect ratio as the correct face location is adopted.

FIG. 5 is a diagram showing an example of the results of generating weathering and deterioration evaluation points for a face from a sketch base image decomposed into evaluation areas.
Referring to Figure 5, the evaluation points generated for the two example sketch base images of Figures 5(a) and (b) decomposed into the evaluation areas of the top, left shoulder, and right shoulder are shown. In the sketch base image of Figure 5(a), the parts shown with intermediate darkness are displayed in gray in the color display, and the darkest parts are displayed in brown in the color display. In the sketch base image of Figure 5(b), almost the entire surface is displayed in brown in the color display.

When generating a weathering evaluation score, the face evaluation unit 18 performs the evaluation based on the area of a pre-specified color. In this embodiment, the color specified is mainly brown, and the evaluation score is determined according to the area of the brown color. In the example of Figure 5(a), the top end where the brown part exists has an evaluation score of 2, and the left shoulder and right shoulder which do not include the brown part have an evaluation score of 1. In the example of Figure 5(b), almost the entire surface is brown, so the evaluation score is set to 4 for the entire evaluation area.

FIG. 6 is a diagram showing an example of a working face observation record manuscript according to an embodiment of the present invention.
Referring to Figure 6, the face observation record manuscript 40 according to an embodiment of the present invention has a face reference information column 41 and a face observation record column 42, and the face observation record column 42 includes an evaluation category column 43 and a sketch column 44.

In the face reference information column 41, necessary information is extracted from the initial registration data that registers the name of the tunnel, specifications of the tunnel construction, etc. at the start of the tunnel construction and entered.
In the evaluation category column 43, only the item name of row (D), "weathering," is entered as an example, but for other individual evaluation items specified as a format, such as the frequency of cracks and water leakage, the item name and the evaluation criteria for that item (1..., 2..., etc.) are entered. At the right end of the evaluation category column 43, the results of evaluation based on the evaluation criteria are entered, categorized as left shoulder, top end, and right shoulder. For evaluation results generated by the evaluation server 10, the results are entered in the corresponding evaluation item column, and for items not generated by the evaluation server 10, the column is left blank so that the worker can enter the results later.
A sketch base image 45 generated by the evaluation server 10 is pasted in the sketch field 44.

The face observation record manuscript 40 shown in Figure 6 is just one example, and the contents to be recorded in the face observation record manuscript 40 can take various forms depending on the specifications of the client of the tunnel construction work, etc. However, since the face observation record manuscript 40 is generated by recording information that has been generated or extracted in advance by the evaluation server 10 according to the format specified by the evaluation server 10, workers can easily complete the face observation record using the face observation record manuscript 40.

FIG. 7 is a flowchart for explaining a tunnel face evaluation method according to an embodiment of the present invention.
7, in step S700, an operator acquires an image of a face using the mobile terminal 20 and transmits the image to the cloud storage 5. The cloud storage 5 stores the received image of the face (step S710).

When the evaluation server 10 detects that a new face image has been saved in the cloud storage 5, for example through a notification from the cloud storage 5, it acquires the newly saved face image from the cloud storage 5 in step S720.
The evaluation server 10, which has acquired the image of the working face, generates a deformed base image for sketching from the image of the working face by color-coding it according to the overall appearance of the color tone, using an image conversion processing means using deep learning in step S730.

In step S740, the evaluation server 10 generates an evaluation score for the face based on the generated sketch base image. The generation of the evaluation score will be described in detail later with reference to FIG.
The evaluation server 10 generates a face observation record manuscript based on the generated sketch base image and the evaluation points (step S750). The format of the face observation record manuscript is a format that is specified in advance and stored in the storage unit 13. If multiple formats are stored in the storage unit 13, an appropriate format is extracted based on identifiable information such as the tunnel client's name and the tunnel name.
The evaluation server 10 transmits the completed face observation record manuscript with the available information written therein to the cloud storage 5, and the cloud storage 5 stores the transmitted face observation record manuscript (step S760).

When the draft face observation record is saved in cloud storage 5, the worker retrieves the saved draft face observation record from cloud storage 5 (step S770) and completes the face observation record by editing it to add missing evaluation points and comments to the sketch base image (step S780). Finally, the worker transmits the completed face observation record from the mobile terminal 20 to cloud storage 5, and the transmitted face observation record is saved in cloud storage 5 (step S790).

Basically, the completed face observation records are accumulated in the cloud storage 5 according to the above flow, but by referring to the accumulated face observation records in chronological order, it is possible to confirm changes in the face condition over time and as the construction progresses. Although not shown in the figure, in the embodiment, when a new face observation record is saved in step S790, the evaluation server 10 reads out and compares the latest face observation record with the accumulated face observation records from before, and if it is determined that the risk of a disaster such as a skin fall has increased, it generates and transmits warning information to the mobile terminal 20 of the worker to warn of the increased risk. The risk of a disaster is determined, for example, by evaluating whether the face evaluation point has changed to a weaker value than the value of the immediately preceding face observation record in chronological order, or whether the running of the face in the excavation direction has changed to a dip plate appearance. In this way, the tunnel face evaluation system 1 according to the embodiment of the present invention makes it possible to reduce the risk of an accident for a worker working at the face by effectively utilizing the completed face observation records to issue a warning.

FIG. 8 is a flowchart for explaining a tunnel face evaluation point generation method for calculating a weathering deterioration evaluation point in a tunnel face evaluation method according to an embodiment of the present invention.
8, in step S800, the evaluation server 10 detects the face location from the face image acquired in step S720 of FIG 7. Next, in order to check whether the face location has been correctly detected, in step S810, it is determined whether the face location is appropriate. If it is determined that the face location is appropriate, the detected face location is used as is. If it is determined that the face location is not appropriate in step S810, the face location is detected in step S820 based on the sketch base image generated in step S730 of FIG 7.
When a suitable face location is detected, the evaluation server 10 superimposes the sketch base image and the face location in step S830, and then decomposes it into evaluation areas of the top, left shoulder, and right shoulder.
Finally, a weathering evaluation score is calculated for each of the divided evaluation areas according to a pre-designated color (step S840).

The above describes in detail an embodiment of the present invention with reference to the drawings, but the present invention is not limited to the above-described embodiment and can be modified in various ways without departing from the technical scope of the present invention.

1 Tunnel face evaluation system 5 Cloud storage 10 Evaluation server 11, 21 Control unit 12, 22 Input/output unit 13, 23 Memory unit 14, 24 Display unit 15 Sketch base image generation unit 16 Face location detection unit 17 Face location decomposition processing unit 18 Face evaluation unit 19 Face observation record manuscript generation unit 20 Mobile terminal 25 Image acquisition unit 26 Edited data input unit 30 Face image 31, 45 Sketch base image 32-1, 32-2 Face location 40 Face observation record manuscript 41 Face reference information column 42 Face observation record column 43 Evaluation category column 44 Sketch column

Claims (6)

A tunnel face evaluation system including an evaluation server connected to a mobile terminal of a worker via a cloud storage network,
The evaluation server includes:
A sketch base image generating unit that acquires an image of a face acquired by the mobile device and stored in the cloud storage, and generates a sketch base image that is deformed by color-coding the image of the face according to the color tone of the appearance from a global perspective using an image conversion processing means using deep learning;
A face evaluation unit that generates an evaluation point of a face based on the generated sketch base image;
A tunnel face evaluation system characterized by comprising a face observation record manuscript generation unit that generates a face observation record manuscript based on the sketch base image and the evaluation points and stores it in the cloud storage. The evaluation server includes a face location detection unit that acquires an image of the face and detects the face location by an object detection means using deep learning, or acquires the sketch base image and detects the face location from a color analysis of the sketch base image;
The tunnel face evaluation system according to claim 1, further comprising a face location decomposition processing unit that decomposes the face location into each evaluation area of the top, left shoulder, and right shoulder. A tunnel face evaluation method using a tunnel face evaluation system including an evaluation server and a mobile terminal connected via a network via a cloud storage,
Acquiring an image of the face using the mobile terminal of the worker and storing it in the cloud storage;
The evaluation server,
A step of acquiring the image of the face stored in the cloud storage, and generating a deformed sketch base image by color-coding the image of the face according to the overall appearance color tone from the image of the face by an image conversion processing means using deep learning;
generating an evaluation point of the face based on the generated sketch base image;
generating a face observation record manuscript based on the sketch base image and the evaluation points;
A tunnel face evaluation method comprising a step of storing the face observation record manuscript in the cloud storage. The tunnel face evaluation method according to claim 3, further comprising a step of acquiring the face observation record manuscript stored in the cloud storage by the mobile terminal, adding supplements or corrections to the face observation record manuscript, and storing it in the cloud storage as a completed face observation record. The step of generating an evaluation point of a face based on the sketch base image includes:
detecting a face location from the face image or the sketch base image;
A step of decomposing the detected face location into evaluation areas of the top, left shoulder, and right shoulder;
A tunnel face evaluation method according to claim 3, further comprising a step of calculating a weathering deterioration evaluation point based on an area of a preset color for each of the decomposed evaluation areas. The step of detecting the face location includes:
Detecting a face location from the image of the face by an object detection means using deep learning;
determining whether the detected face location is appropriate;
The tunnel face evaluation method described in claim 5, characterized in that it includes a step of adopting the detected face location when it is determined that the detected face location is appropriate, and detecting the face location from the sketch base image and adopting the face location detected from the sketch base image when it is determined that the detected face location is inappropriate.

Images