JP7793568B2 - CAD data generation device, method, and program - Google Patents

Description

The present invention relates to technology for generating CAD data from image data of drawings.

CAD (Computer Aided Design) data has traditionally been generated using images of drawings (see Patent Document 1). Images of drawings contain a mixture of graphics and text. CAD data, on the other hand, is composed of multiple layers that separate graphics and text. Therefore, when generating CAD data from image data of a drawing, the conventional method has been to extract text from the drawing using OCR (Optical Character Recognition) and generate text data, and to manually create raster data for each layer of graphics and convert it into vector data, and then integrate the data from each layer of text and graphics into the CAD data.

Japanese Patent Application Laid-Open No. 2019-133422

With conventional methods, the manual process of creating vector data for each layer was tedious. Furthermore, text in drawings was often written at an angle or drawn close to lines, resulting in poor accuracy in text extraction using OCR.

One objective of this disclosure is to provide technology that makes it possible to easily generate CAD data from images of drawings.

A CAD data generation device according to one aspect of the present disclosure is a CAD data generation device that generates CAD data including multiple layer data from an image of a drawing. The CAD data generation device has a memory and a processor, where the memory stores an inference model for identifying layer areas, which are areas having the characteristics of each layer, from the image, and the processor uses the inference model to identify the layer areas of each layer from the image of the drawing and generates CAD data including data of the layer areas as layer data for each layer.

One aspect of the present disclosure provides technology that makes it possible to easily generate CAD data from an image of a drawing.

FIG. 1 is a diagram showing a configuration of general CAD data. FIG. 1 is a functional block diagram of a CAD data generation device according to an embodiment of the present disclosure. FIG. 2 is a conceptual diagram illustrating various data stored in a storage unit of a CAD data generation device according to an embodiment of the present disclosure. 1 is a flowchart illustrating a process for implementing a CAD data generation method by a CAD data generation device according to an embodiment of the present disclosure. FIG. 10 is a diagram showing a state in which a layer area decomposition unit of a CAD data generation device according to an embodiment of the present disclosure performs a process of dividing the image data of a drawing into image data for each layer area, and each layer area included in the image data of the drawing is identified. 10A is a diagram showing image data of a figure layer area and image data of a drawing frame layer area obtained by a layer area decomposition unit dividing the image data of a drawing into image data for each layer area; FIG. This figure shows image data (c) of the dimension line sub-layer area, image data (d) of the dimension character sub-layer area, and image data (e) of the character layer area obtained by the layer area decomposition unit dividing the image data of the drawing into image data for each layer area.

Embodiments of the present invention will be described below with reference to the drawings.

First, the structure of typical CAD data will be described with reference to Figure 1. As shown in Figure 1, CAD data related to design drawings for mechanical structures, electrical circuit wiring, etc., typically has a structure in which multiple data layers are superimposed. The multiple data layers of CAD data include, for example, a graphics layer containing line drawings of the design object; a drawing frame layer containing the outer frame and ruled lines of the design drawing; a dimension layer containing dimension lines and numerical dimension text indicating the dimensions of each part of the design object drawn in the graphics layer; and a text layer containing text written in the design drawing. Of the above data layers, the dimension layer may be composed of a dimension line sublayer containing dimension lines and a dimension text sublayer containing numerical dimension text. CAD data typically includes these multiple superimposed data layers. These data layers that may be included in CAD data are exemplary; some of these data layers may be omitted, or data layers containing other information may be added.

Next, a CAD data generation device according to an embodiment of the present disclosure will be described. FIG. 2 is a functional block diagram of a CAD data generation device according to an embodiment of the present disclosure. As shown in FIG. 2, the CAD data generation device 100 of this embodiment includes a control unit 110, a storage unit 120, and an input/output interface 130.

The control unit 110 controls the processing operations of the CAD data generation device 100 by executing programs stored in the memory unit 120, and is composed of one or more CPUs (Central Processing Units) and GPUs (Graphics Processing Units). The control unit 110 includes, as functional components, an image input unit 111, a layer region decomposition unit 112, a data conversion unit 113, a sub-layer combination unit 114, and a CAD data output unit 115. Each of these functional components included in the control unit 110 operates to implement the processing and functions performed by the CAD data generation device 100, as described in particular with reference to FIG. 4, etc.

The image input unit 111 performs processing to receive image data of drawings input via the input/output interface 130 (described below) from an external terminal of the CAD data generation device 100 or from other functional components within the CAD data generation device 100. The image data of drawings input from an external terminal, etc., can be in any file format, such as JPEG or PNG.

The layer region decomposition unit 112 performs processing to divide the image data received by the image input unit 111 into image data for each of the above-mentioned layer regions. In this embodiment, the layer region decomposition unit 112 is configured to perform processing to divide the image data into regions for each layer using an inference model 122 stored in the storage unit 120, as described below.

The data conversion unit 113 performs processing to convert or extract the image data of each layer area divided into layers by the layer area decomposition unit 112 in a manner suited to each layer area.

More specifically, for graphic layer areas containing line drawings for depicting design objects, the data conversion unit 113 converts the raster format data contained in the graphic layer areas divided by the layer area decomposition unit 112 into vector format data, and for drawing frame layer areas containing the outer frames and ruled lines of design drawings, the data conversion unit 113 converts the raster format data contained in the drawing frame layer areas divided by the layer area decomposition unit 112 into vector format data.

The data conversion unit 113 also converts raster data relating to dimension lines contained in the dimension line sub-layer area after division by the layer area decomposition unit 112 into vector data for the dimension layer containing dimension lines and dimension numeric characters indicating the dimensions of each part of the design object drawn on the graphic layer, and extracts character data of the dimension numeric characters contained in the dimension character sub-layer area after division using any character recognition process such as OCR. Furthermore, for character layers containing characters written in the design drawings, the character data contained in the character layer area after division by the layer area decomposition unit 112 is extracted using OCR.

The sublayer combining unit 114 performs a process to combine multiple sublayer data to generate a single layer data for a layer that includes multiple sublayers, such as a dimension layer that includes a dimension line sublayer containing dimension lines and a dimension text sublayer containing dimension numerical values (text).

The CAD data output unit 115 is configured to execute a process of integrating multiple layer data generated by the data conversion unit 113 and the sub-layer combining unit 114 to generate a single piece of CAD data. The CAD data output unit 115 outputs the generated CAD data via the input/output interface 130 to an external terminal of the CAD data generation device 100 or to other functional components within the CAD data generation device 100.

The memory unit 120 stores programs, input data, intermediate data, etc. for executing the various control processes and functions of the control unit 110, and includes main memory including RAM (Random Access Memory) and ROM (Read Only Memory), as well as storage memory such as HDD (Hard Disk Drive) and SSD (Solid State Drive). Furthermore, the memory unit 120 stores an inference model 122 that the control unit 110 uses to divide the image data of drawings input to the control unit 110 from an external terminal or functional configuration unit into the above-mentioned layer regions, as described below.

The inference model 122 is an inference model generated by machine learning using image data from various drawings as training data. The training data used to generate the inference model 122 may be, for example, multiple image data including annotation information labeling each layer area within the drawing (figure layer area, drawing frame layer area, dimension line sub-layer area, dimension text sub-layer area, text layer area), or multiple CAD data that already contain data for each of the above-mentioned layer areas, each individually labeled. The inference model 122 can be created using any external computer device (not shown) as a learner and stored in the memory unit 120.

In this embodiment, the inference model 122 can be generated using a deep learning algorithm known as semantic segmentation, which associates a label or category with every pixel in an image. The inference model generated by semantic segmentation can be used to recognize groups of pixels that form distinctive categories. In particular, in this embodiment, the inference model generated by semantic segmentation can be used to recognize groups of pixels that form each layer area (figure layer area, drawing frame layer area, dimension line sublayer area, dimension text sublayer area, and text layer area) included in the image data of a drawing.

As an example, the inference model 122 is generated by performing machine learning on a neural network consisting of multiple layers, each containing neurons. Such a neural network can be a deep neural network such as a convolutional neural network (CNN).

The memory unit 120 also at least temporarily stores various data generated by the control unit 110 when it executes processing (including intermediate data generated during processing).

Figure 3 is a conceptual diagram showing the various data stored in the storage unit 120. As shown in Figure 3, the storage unit 120 stores, as drawing image data, data of drawing images input via the input/output interface 130 from an external terminal of the CAD data generation device 100 or from other functional components within the CAD data generation device 100.

The storage unit 120 also stores, as image data for each layer area, image data for the figure layer area, drawing frame layer area, dimension line layer area, dimension value layer area, and text layer area that have been individually separated from the drawing image data by the layer area decomposition unit 112. The data for each of these layer areas stored as image data for each layer area is raster format data before the data conversion or extraction process by the data conversion unit 113 is performed.

The storage unit 120 also stores, as CAD layer-specific data, drawing layer data, drawing frame layer data, dimension layer data, and character layer data generated by the data conversion unit 113 and sub-layer combining unit 114 through data conversion or extraction and sub-layer data combining processes. Each of these layer data stored as CAD layer-specific data is vector-format data after data conversion or extraction by the data conversion unit 113, or character data extracted by OCR. The CAD data output unit 115 integrates each of these layer data stored as CAD layer-specific data into a single CAD data and outputs it.

The input/output interface 130 is an interface for transferring data between the CAD data generation device 100 and an external terminal or other functional components within the CAD data generation device 100. Devices that comply with various standards for data transfer can be used as the input/output interface 130.

Next, a CAD data generation method using the CAD data generation device according to this embodiment will be described with reference to Figure 4 and other figures. Figure 4 is a flowchart showing the processing for implementing the CAD data generation method using the CAD data generation device according to this embodiment.

In the CAD data generation method of this embodiment, first, the image input unit 111 of the control unit 110 of the CAD data generation device 100 executes a process to receive image data of a drawing input via the input/output interface 130 from an external terminal of the CAD data generation device 100 or from another functional component within the CAD data generation device 100 (step S101). The image data of the drawing input from an external terminal or the like can be in any file format, such as JPEG or PNG.

Next, the layer region decomposition unit 112 of the control unit 110 performs a process of dividing the image data received by the image input unit 111 into image data for each layer region based on the inference model 122 stored in the memory unit 120 (step S102).

As mentioned above, examples of image data for each layer area include the graphic layer area, drawing frame layer area, dimension line sub-layer area, dimension text sub-layer area, and text layer area. This image data is stored in raster format in the storage unit 120, at least temporarily, as described with reference to Figure 3.

Figure 5 shows the state in which the layer region decomposition unit 112 has performed a process to divide the image data of the drawing into image data for each layer region, and each layer region contained in the image data of the drawing has been identified. The dotted lines shown in Figure 5 are shown to illustrate the boundaries that define each layer region, and do not appear in the image after processing by the layer region decomposition unit 112.

Figures 6 and 7 each show the image data for each region obtained when the layer region decomposition unit 112 divides the image data of the drawing into image data for each layer region. More specifically, (a) in Figure 6 shows the image data for the figure layer region, (b) in Figure 6 shows the image data for the drawing frame layer region, (c) in Figure 7 shows the image data for the dimension line sub-layer region, (d) in Figure 7 shows the image data for the dimension text sub-layer region, and (e) in Figure 7 shows the image data for the text layer region. The dotted lines shown in Figures 6 and 7 are also shown to illustrate the boundaries that define each layer region, and are not included in the image data for each layer region.

Next, the control unit 110 determines whether image data for all layer areas has been generated from the image data of the drawing received in step S101 (step S103). If it is determined that image data for all layer areas has not been generated (NO), the process proceeds to step S104; if it is determined that image data for all layer areas has been generated (YES), the process proceeds to step S107.

If it is determined in step S103 that image data for all layer areas has not been generated (NO), the control unit 110 determines whether the image data to be processed is image data for a dimension character sub-layer area or image data for a character layer area (step S104). If it is determined that the image data to be processed is image data for a dimension character sub-layer area or image data for a character layer area (YES), processing proceeds to step S105. If it is determined that the image data to be processed is not image data for a dimension character sub-layer area or image data for a character layer area (NO), processing proceeds to step S106.

If step S104 determines that the image data to be processed is image data related to a dimension character sub-layer area or image data related to a character layer area (YES), the data conversion unit 113 of the control unit 110 extracts character data from the image data using any character recognition process such as OCR to generate dimension character sub-layer data if the image data to be processed is image data related to a dimension character sub-layer area, or extracts character data from the image data using OCR to generate character layer data if the image data to be processed is image data related to a character layer area (step S105). This dimension character sub-layer data and character layer data are at least temporarily stored in the storage unit 120 as CAD layer-specific data, as described with reference to FIG. 3.

On the other hand, if it is determined in step S104 that the image data to be processed is not image data relating to a dimension character sub-layer area or image data relating to a character layer area (NO), the data conversion unit 113 of the control unit 110 converts the raster format data contained in the image data of that layer area into vector format data, and generates layer data for each layer area (step S106).

More specifically, in this embodiment, the data conversion unit 113 converts the raster format data contained in the image data of the figure layer area, drawing frame layer area, and dimension line sub-layer area into vector format data, thereby generating figure layer data, drawing frame layer data, and dimension line sub-layer data, respectively. The figure layer data, drawing frame layer data, and dimension line sub-layer data generated as vector format data are at least temporarily stored in the storage unit 120 as CAD layer-specific data, as described with reference to FIG. 3.

Next, in step S107, the control unit 110 determines whether generation of all layer data has been completed. In the example of this embodiment, if the CAD data to be generated includes graphic layer data, drawing frame layer data, dimension layer data, and text layer data, the control unit 110 determines whether generation of all of this layer data has been completed. If it is determined that generation of all layer data has been completed (YES), processing proceeds to step S110; if it is determined that generation of all layer data has not been completed (NO), processing proceeds to step S108.

If it is determined in step S107 that generation of all layer data has not been completed (NO), the control unit 110 determines whether the data determined to be layer data for which generation has not been completed is dimension character sub-layer data or dimension line sub-layer data (step S108). If it is determined that the data determined to be layer data for which generation has not been completed is dimension character sub-layer data or dimension line sub-layer data (YES), processing proceeds to step S109; if not (NO), processing returns to step S107.

If the data determined to be layer data for which generation has not been completed in step S108 is determined to be dimension text sub-layer data or dimension line sub-layer data (YES), the sub-layer combining unit 114 of the control unit 110 combines the dimension text sub-layer data and dimension line sub-layer data into the dimension layer data, completing generation of the dimension layer data (step S109). This completes the generation of all layer data included in the CAD data to be generated: graphic layer data, drawing frame layer data, dimension layer data, and character layer data.

If it is determined in step S107 that generation of all layer data has been completed (YES), processing proceeds to step S110, where the CAD data output unit 115 of the control unit 110 executes processing to integrate all of the layer data to generate one piece of CAD data. The CAD data output unit 115 outputs the generated CAD data via the input/output interface 130 to an external terminal of the CAD data generation device 100 or to another functional component within the CAD data generation device 100.

As described above, one aspect of the present disclosure provides a CAD data generation device that generates CAD data including multiple layer data from an image of a drawing. The CAD data generation device has a memory and a processor, where the memory stores an inference model for identifying layer regions, which are regions having characteristics of each layer, from the image, and the processor uses the inference model to identify the layer regions of each layer from the image of the drawing and generates CAD data including data of the layer regions as layer data for each layer.

The CAD data generation device disclosed herein, configured in this manner, uses an inference model to identify areas with characteristics of each layer and generates CAD data that includes layer data for those layer areas, making it possible to easily generate CAD data from drawing images.

The CAD data generation device of the present disclosure may further be configured such that the inference model is a semantic segmentation model that divides an image into layer regions for each layer. This allows each layer region to be divided using a semantic segmentation model obtained by machine learning from image data of a drawing that includes various layer regions, eliminating the need for region division using image processing that requires delicate setting of thresholds.

The CAD data generation device of the present disclosure may further include a CAD data layer in which dimensions are indicated by dimension lines and dimension text, an inference model that can identify a dimension line sub-layer region having the characteristics of the dimension lines and a dimension text sub-layer region having the characteristics of the dimension text, and a processor that can identify the dimension line sub-layer region and the dimension text sub-layer region from the image of the drawing and combine the data of the dimension text sub-layer region and the data of the dimension line sub-layer region into the dimension layer. This allows the lines and text representing dimensions to be separated by different processes and identified by the inference model, making it possible to separately recognize dimension lines and dimension text written near them, and thus allowing layer data for a dimension layer that includes a dimension line sub-layer region and a dimension text sub-layer region to be appropriately created.

The CAD data generation device of the present disclosure may further be configured such that the CAD data includes a drawing frame layer in which the drawing frame is indicated, and the inference model can identify a drawing frame layer region having the characteristics of the drawing frame, and the processor can identify the drawing frame layer region from an image of the drawing, convert raster data of the image of the drawing frame layer region into vector data, and create the vector data as a drawing frame layer. This makes it possible to identify the drawing frame layer region based on the inference model from image data of a drawing that includes various layer regions, and generate the drawing frame layer as vector data.

The CAD data generation device of the present disclosure may further be configured such that the CAD data includes a graphic layer containing graphics included in the drawing, and the inference model can identify a graphic layer region having the characteristics of the graphic, and the processor can identify the graphic layer region from an image of the drawing, convert raster data of the image of the graphic layer region into vector data, and convert the vector data into a graphic layer. This makes it possible to identify a graphic layer region based on the inference model from image data of a drawing containing various layer regions, and generate the graphic layer as vector data.

The CAD data generation device of the present disclosure may further be configured such that the CAD data includes a character layer containing characters included in the drawing, the inference model can identify a character layer area having the characteristics of the character, and the processor can identify the character layer area from the image of the drawing, extract character data from the image of the character layer area using character recognition processing, and convert the character data into a character layer. This makes it possible to identify a character layer area based on the inference model from image data of a drawing containing various layer areas, and generate the character layer as character data.

According to another aspect of the present disclosure, there is provided a method for generating CAD data including multiple layer data from an image of a drawing, the method being executed by a CAD data generation device having a memory and a processor, wherein the memory stores an inference model for identifying layer regions, which are regions having characteristics of each layer, from the image, and the method includes the processor using the inference model to identify layer regions of each layer from the image of the drawing, and the processor generating CAD data including data of the layer regions as layer data for each layer.

According to yet another aspect of the present disclosure, there is provided a program for causing a computer having a processor and memory to generate CAD data including multiple layer data from an image of a drawing, wherein the memory stores an inference model for identifying layer regions, which are regions having characteristics of each layer, from the image, and the program causes the processor to identify the layer regions of each layer from the image of the drawing using the inference model and generate CAD data including data of the layer regions as layer data for each layer.

The above-described embodiments of the present invention are illustrative examples of the present invention and are not intended to limit the scope of the present invention to these embodiments alone. Those skilled in the art may implement the present invention in various other forms without departing from the scope of the present invention.

100...CAD data generator, 110...control unit, 120...storage unit, 130...input/output interface

Claims (7)

A CAD data generation device that generates CAD data including a plurality of layer data including a dimension layer that indicates dimensions using dimension lines and dimension characters from an image of a drawing,
a memory and a processor;
The memory stores an inference model for identifying a layer area, which is an area having the characteristics of each layer, from an image, and capable of identifying a dimension line sub-layer area having the characteristics of the dimension line and a dimension character sub-layer area having the characteristics of the dimension character ;
The processor:
Using the inference model, a layer area of each layer is identified from the image of the drawing, and CAD data including data of the layer area as layer data of each layer is generated;
For the dimension layer, the inference model is used to identify the dimension line sub-layer area and the dimension character sub-layer area from the image of the drawing, raster data of the image of the dimension line sub-layer area is converted into vector data, character data is extracted from the image of the dimension character sub-layer area by character recognition processing, and the vector data and the character data are combined into the dimension layer.
CAD data generator. The inference model is a semantic segmentation model that divides an image into layer regions of each layer.
2. The CAD data generating device according to claim 1. The CAD data includes a drawing frame layer on which a drawing frame is displayed,
the inference model is capable of identifying a drawing border layer region having the characteristics of the drawing border;
The processor:
Identifying the drawing frame layer area from the image of the drawing;
converting raster data of the image in the drawing frame layer area into vector data;
the vector data into the drawing frame layer;
2. The CAD data generating device according to claim 1. The CAD data includes a figure layer including figures included in the drawing,
the inference model is capable of identifying a graphic layer region having the characteristics of the graphic;
The processor:
Identifying the graphics layer area from the image of the drawing;
converting raster data of the image in the graphics layer area into vector data;
converting the vector data into the graphic layer;
2. The CAD data generating device according to claim 1. The CAD data includes a character layer including characters included in the drawing,
the inference model is capable of identifying a character layer region having the characteristics of the character;
The processor:
Identifying the character layer area from the image of the drawing;
extracting character data from the image of the character layer area by character recognition processing;
the character data is converted into the character layer;
2. The CAD data generating device according to claim 1. A method for generating CAD data including a plurality of layer data including a dimension layer indicating dimensions by dimension lines and dimension characters from an image of a drawing, the method being executed by a CAD data generating device having a memory and a processor, the method comprising:
The memory stores an inference model for identifying a layer area, which is an area having the characteristics of each layer, from an image, and capable of identifying a dimension line sub-layer area having the characteristics of the dimension line and a dimension character sub-layer area having the characteristics of the dimension character ;
The method comprises:
the processor using the inference model to identify a layer region for each layer from the image of the drawing;
The processor generates CAD data including data of the layer region as layer data of each layer;
Including ,
For the dimension layer, the processor uses the inference model to identify the dimension line sublayer area and the dimension character sublayer area from the image of the drawing, converts raster data of the image of the dimension line sublayer area into vector data, extracts character data from the image of the dimension character sublayer area by character recognition processing, and combines the vector data and the character data into the dimension layer.
method. A program that causes a computer having a processor and a memory to generate CAD data including a plurality of layer data including a dimension layer that indicates dimensions using dimension lines and dimension characters from an image of a drawing,
The memory stores an inference model for identifying a layer area, which is an area having the characteristics of each layer, from an image, and capable of identifying a dimension line sub-layer area having the characteristics of the dimension line and a dimension character sub-layer area having the characteristics of the dimension character ;
The program causes the processor to:
Identifying a layer region of each layer from the image of the drawing using the inference model;
generating CAD data including the data of the layer area as layer data of each layer;
Execute
Regarding the dimension layer, the inference model is used to identify the dimension line sub-layer area and the dimension character sub-layer area from the image of the drawing, raster data of the image of the dimension line sub-layer area is converted into vector data, character data is extracted from the image of the dimension character sub-layer area by character recognition processing, and the vector data and the character data are combined into the dimension layer.
Let it run,
program.