JP7303676B2 - Equipment detection device - Google Patents

Description

TECHNICAL FIELD Embodiments of the present invention relate to a device detection device and a device detection method.

A technique is known for detecting an object registered in a database by detecting a two-dimensional code existing in an image.

Japanese Patent No. 4724800 JP 2018-16489 A

However, the above technique poses a problem in that it is necessary to install the camera at a distance close to the object. In this case, it is necessary to install many cameras in order to detect objects installed over a wide range.

Accordingly, an object of the embodiments of the present invention is to provide a device detection device and a device detection method capable of efficiently detecting objects installed over a wide range.

According to one embodiment, a device detection device includes an imaging unit that acquires a captured image captured by an imaging device that moves inside a building. The device detection apparatus further includes a detection unit that detects, from the captured image, a device identification graphic, which is a graphic provided on or near the device for identifying the device in the building. The device detection device further compares the first identification information obtained from the device identification graphic with the second identification information stored in the storage unit as the identification information of the device to obtain the first identification information. An extraction unit for extracting the second identification information corresponding to the identification information from the storage unit. The device detection device further includes information output from the imaging device or output from a device moving together with the imaging device when the second identification information corresponding to the first identification information is extracted. A determining unit for determining location information indicating the location of the device within the building based on the information. The device detection apparatus further includes a registration unit that associates and registers the identification information of the device and the position information determined by the determination unit.

1 is a block diagram showing the configuration of a device detection system according to a first embodiment; FIG. 1 is a block diagram showing the configuration of a device detection device according to a first embodiment; FIG. It is a figure which shows the structure of the apparatus detection system of 1st Embodiment. 4A and 4B are diagrams showing an example of a captured image output from the imaging device of the first embodiment and an example of a device marker of the first embodiment; FIG. It is a figure which shows the example of the captured image output from the imaging device of 1st Embodiment. It is a figure which shows the example of the stored data in the database of 1st Embodiment. 4 is a flowchart for explaining a device detection method according to the first embodiment; 4 is a flowchart for explaining details of a device detection method according to the first embodiment; It is a figure which shows the structure of the apparatus detection system of the 1st modification of 1st Embodiment. It is a figure which shows the example of the captured image output from the imaging device of the 1st modification of 1st Embodiment. It is a figure which shows the example of the position coordinate table of the 1st modification of 1st Embodiment. 9 is a flowchart for explaining a device detection method according to a first modified example of the first embodiment; It is a block diagram which shows the structure of the apparatus detection apparatus of 2nd Embodiment. 8 is a flowchart for explaining a device detection method according to the second embodiment; It is a block diagram which shows the structure of the apparatus detection apparatus of 3rd Embodiment. FIG. 11 is a diagram showing an example of a connected image according to the third embodiment; FIG. 10 is a flowchart for explaining a device detection method according to the third embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. 1 to 17, the same or similar configurations are denoted by the same reference numerals, and redundant description is omitted.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of the device detection system of the first embodiment.

The device detection system of the present embodiment includes a device detection device 1, a device information management device 2, and one or more imaging devices 3, as shown in FIG. FIG. 1 shows three imaging devices 3a to 3c as an example of one or more imaging devices 3. FIG. FIG. 1 further shows a device marker 4 and a position marker 5 to be imaged by the imaging device 3 and a crane 6 for moving the device detection device 1 and the imaging device 3 . The equipment marker 4, the position marker 5, and the crane 6 are examples of the equipment identification graphic, the position identification graphic, and the moving object, respectively. Details of the equipment marker 4, the position marker 5, and the crane 6 will be described later.

As will be described later, the device detection system of this embodiment is provided in a building in which a plurality of devices are arranged, and operates to detect these devices. An example of a building is a building built at a manufacturing site for manufacturing industrial products, and an example of equipment is a control panel for manufacturing industrial products. The device detection device 1 acquires captured images captured inside the building by each imaging device 3, detects devices based on the device markers 4 in the captured images, and detects devices based on the position markers 5 in the captured images. Detect location. The device information management device 2 manages information necessary for the operation of the device detection device 1 and information output from the device detection device 1 .

The device detection device 1 is, for example, a computer such as a PC (Personal Computer) or a server device. The device detection device 1 includes input/output devices 11 such as keyboards, mice, displays, and communication devices, arithmetic devices 12 such as processors and ROMs (Read Only Memories), and storage devices such as HDDs (Hard Disc Drives) and semiconductor memories. 13 and a program 14 in the storage device 13 . The device detection device 1 can communicate with each imaging device 3 through wired communication or wireless communication.

The device information management device 2 is, for example, a computer such as a PC or a server device. The device information management device 2 includes an arithmetic device 21 such as a processor or ROM, a storage device 22 such as an HDD or semiconductor memory, and a database 23 . The database 23 may be included in the storage device 22 or may be provided separately from the storage device 22 . Also, the database 23 may be composed of one device, or may be composed of a plurality of devices. The database 23 is an example of a storage unit. The device information management device 2 can communicate with the device detection device 1 through wired communication or wireless communication. The device detection device 1 and the device information management device 2 of this embodiment are connected to each other via a network.

FIG. 2 is a block diagram showing the configuration of the device detection device 1 of the first embodiment.

As shown in FIG. 2, the device detection device 1 of the present embodiment includes an imaging unit 31, a marker detection unit 32, an identification unit 33, a collation unit 34, a position estimation unit 35, a registration unit 36, and a storage unit. a portion 37; These blocks are implemented, for example, by executing the program 14 described above in the device detection device 1 . Details of these blocks will be described later. The identifying unit 33 and the matching unit 34 are examples of the extracting unit, and the position estimating unit 35 is an example of the determining unit.

FIG. 3 is a diagram showing the configuration of the device detection system of the first embodiment.

FIG. 3 is a perspective view showing the overall configuration of the device detection system of the present embodiment. The floor 41 of the building described above, the passage 42 provided in the center of the floor 41, and a plurality of sensors arranged on the floor 41 are shown in FIG. equipment 43 and a vehicle 44 carrying the equipment 43 are shown. An example of equipment 43 is a control panel and an example of vehicle 44 is a truck. FIG. 3 also shows the x and y directions parallel to and perpendicular to the floor 41 and the z direction perpendicular to the floor 41 .

In this embodiment, as the manufacturing process of a certain product progresses, the installation location of the device 43 used for manufacturing this product is moved. In this embodiment, the installation location of each device 43 is managed by the device detection device 1 or the like. As a result, it is possible to reduce the time and effort required to manage the equipment 43, and to suppress delays in the manufacturing work due to delays in managing the equipment 43.

FIG. 3 further shows a device detection device 1, a device information management device 2, an imaging device 3 (imaging devices 3a to 3c), a plurality of device markers 4, a plurality of position markers 5, and a crane 6. there is

The device detection device 1 and the imaging device 3 are attached to a crane 6 that can move inside the building. FIG. 3 shows the crane 6 about to move above the floor 41 in the y-direction. On the other hand, the device information management device 2 is installed in a stationary state inside the building. The imaging device 3 may be moved inside the building by means other than the crane 6 .

Note that the device information management device 2 may be installed in a building other than the building shown in FIG. Such a configuration can be employed, for example, when managing information on the equipment 43 in a plurality of buildings with this single equipment information management device 2 . In this case, it is conceivable to dispose the device detection device 1 in each building and configure a device detection system with a plurality of device detection devices 1 and one device information management device 2 .

The device marker 4 is a mark used to identify the device 43 and is provided on each device 43 . In this embodiment, one device marker 4 printed on paper is attached to the top of each device 43 . Equipment markers 4 holding identifiers different from each other are affixed to the top of a certain equipment 43 and the top of another equipment 43 in the building. The device marker 4 may be any figure as long as it can hold such an identifier, such as a one-dimensional bar code, a two-dimensional bar code, or a QR code. Also, the graphics referred to here may be one or more letters, numbers, or symbols, or a combination thereof. The device marker 4 of this embodiment is a color marker in which one of a plurality of types of colors is arranged in each of m×n divided cells.

The device marker 4 of this embodiment is provided above the device 43 to be managed as described above. However, if the device 43 to be managed does not have a ceiling plate, the device marker 4 of the device 43 may be placed on the desk near the device 43 . The direction of the device marker 4 may be set in any direction. For example, the long side of the device marker 4 may be parallel to the x direction, or the long side of the device marker 4 may be parallel to the y direction. In addition, since the device marker 4 is an imaging target of the imaging device 3 , it is arranged so that a shield that hinders imaging does not occur between the device marker 4 and the imaging device 3 . The surface of the device marker 4 may be processed to suppress reflection of light. Further details of the instrument marker 4 are provided below.

The position markers 5 are marks used to identify positions within the building, and are provided at multiple locations within the building. The position marker 5 of this embodiment is printed on paper and pasted on the passage 42 of the building. In this embodiment, by combining the detection result of the device marker 4 and the detection result of the position marker 5, it is possible to determine (estimate) where each device 43 is installed in the building. Position markers 5 carrying different identifiers are affixed to one location and another location in the building. The position marker 5 may be any figure as long as it can hold such an identifier, such as a one-dimensional bar code, two-dimensional bar code, QR code, or AR code. Also, the graphics referred to here may be one or more letters, numbers, or symbols, or a combination thereof. The position marker 5 of this embodiment is a color marker like the device marker 4 .

The orientation of the position marker 5 may be set in any way. In addition, since the position marker 5 is an object to be imaged by the imaging device 3 , it is arranged so that there is no shielding between the position marker 5 and the imaging device 3 that hinders imaging. The surface of the position marker 5 may be processed to suppress reflection of light. The position markers 5 may be arranged at regular intervals, or may be arranged at irregular intervals. Each position marker 5 may be imaged by any of the imaging device 3a, the imaging device 3b, and the imaging device 3c. Further details of the position marker 5 will be described later.

The imaging device 3 is attached to the crane 6 at an angle that allows imaging of the equipment marker 4 and the position marker 5 . The imaging device 3 of this embodiment is attached to the crane 6 at an angle facing the floor 41 . The image pickup devices 3a to 3c may image areas that overlap each other. The imaging device 3 is, for example, a CCTV (Closed-Circuit Television) camera, but may be another imaging device. The imaging device 3 may be a device that captures moving images or a device that captures still images.

FIG. 3 shows three imaging devices 3a to 3c as examples of the imaging device 3. As shown in FIG. The device detection system of the present embodiment includes a plurality of imaging devices 3 in order to cover all possible ranges of installation of the device markers 4 and the position markers 5 as an imaging range. The number of imaging devices 3 in the device detection system may be set to any number according to the resolution of the imaging device 3 and the installation range of the device marker 4 and the position marker 5 . Therefore, the number of imaging devices 3 in the device detection system may be other than three.

Here, it should be noted that the imaging devices 3a to 3c of this embodiment are moved by the crane 6 inside the building. These imaging devices 3a to 3c cannot image the entire installation range of the device marker 4 and the position marker 5 while standing still at one place. However, by moving these imaging devices 3a to 3c within the building, it is possible to image the entire installation range of the device marker 4 and the position marker 5. FIG.

The device detection device 1 of the present embodiment acquires a captured image captured by the imaging device 3, detects the device marker 4 and the position marker 5 from the captured image, and detects the detection result of the device marker 4 and the position marker 5. The installation position of each device 43 is determined (estimated) by combining the results. Before this determination, the identifier obtained from the device marker 4 and the identifier in the database 23 are compared, and the identifier obtained from the position marker 5 and the identifier in the database 23 are compared. An example of the identifier obtained from the equipment marker 4 is the registration number of the equipment 43, and an example of the identifier obtained from the position marker 5 is information about the compartments in the building. The result of the above determination is registered in the database 23 and the information in the database 23 is updated. In this manner, the device detection system of this embodiment can automatically manage the position of each device 43 .

Examples of captured images captured by the imaging device 3 are the captured image 7a in FIG. 4A, the captured image 7b in FIG. 5A, the captured image 7c in FIG. 5B, and the captured image 7c in FIG. 10(a), the captured image 7e of FIG. 10(b), and the like. These figures show the X direction (X coordinate) corresponding to the x direction inside the building, and the Y direction (Y coordinate) corresponding to the y direction inside the building. The device detection system of the present embodiment manages the positions of the device marker 4 and the position marker 5 using X coordinates and Y coordinates. Hereinafter, when an arbitrary captured image such as these captured images 7a to 7e is described as an example, this will be referred to as "captured image 7".

Hereinafter, details of the imaging unit 31, the marker detection unit 32, the identification unit 33, the matching unit 34, the position estimation unit 35, the registration unit 36, and the storage unit 37 of the device detection apparatus 1 will be described with reference to FIG. In this description, FIGS. 4 to 6 are also referred to as appropriate.

The imaging unit 31 acquires the captured image 7 captured by the imaging device 3 . The imaging device 3 of this embodiment moves to a plurality of locations in the building together with the crane 6 and outputs the captured images 7 captured at these locations to the imaging unit 31 . The imaging unit 31 may store the captured image 7 in the semiconductor memory, or may store it in the HDD.

The marker detection unit 32 detects the area where the device marker 4 exists and the area where the position marker 5 exists from the captured image 7 . The device marker 4 of this embodiment is composed of m×n cells and an outer frame surrounding these cells. Each cell is filled with one of six colors (red, blue, green, yellow, cyan, magenta). The outer frame is filled with a color separable from the background color. The position marker 5 of this embodiment also has the same configuration as the device marker 4 . The marker detection unit 32 detects the device marker 4 and the position marker 5 from the captured image 7 by an image processing method such as color extraction.

The device marker 4 holds an identifier of the device 43 (registration number of the device 43) by the color of the cells forming the device marker 4. FIG. The identifier of the device 43 is also stored in the database 23 and used to match the identifier obtained from the device marker 4 . The identifier held by the device marker 4 is an example of first identification information, and the identifier stored in the database 23 for matching is an example of second identification information.

Similarly, the position marker 5 holds the identifier of the position within the building (information about the section within the building) by the color of the cells forming the position marker 5 . Identifiers of locations within the building are also stored in database 23 and are used to match identifiers obtained from location markers 5 . The identifier held by the position marker 4 is an example of third identification information, and the identifier stored in the database 23 for matching is an example of fourth identification information.

FIG. 4 is a diagram showing an example of the captured image 7 output from the imaging device 3 of the first embodiment and an example of the device marker 4 of the first embodiment.

FIG. 4A shows a captured image 7a output from one of the imaging devices 3 as an example of the captured image 7. FIG. The captured image 7a shows two devices 43 (devices 43a and 43b) placed on the floor 41 and two device markers 4 (device markers 4a and 4b) pasted on the upper portions of these devices 43. contains. FIG. 4(b) is an enlarged view showing the device marker 4a.

The marker detection unit 32 extracts an image range that matches the extraction target color from the captured image 7a. The marker detection unit 32 may use the RGB color space or the HSV color space as a color identification method used to extract the image range. The marker detection unit 32 generates an image by converting the image range other than the extraction target color to black from the captured image 7a. The marker detection unit 32 of the present embodiment performs image processing such as binarization and noise removal on the converted image in order to specify the image range of the device markers 4a and 4b with higher accuracy. The marker detection unit 32 of the present embodiment further calculates the area of the image range (rectangular area) using the image after image processing, and identifies the device markers 4a and 4b that fall within the range of the threshold.

The marker detection unit 32 detects the device marker 4 by extracting color information from the captured image 7a. The marker detection unit 32 recognizes that the color matching the extraction target color stored in the storage unit 37 is the color of the outer frame of the device marker 4 . In addition, the storage unit 37 also stores a threshold value for determining the range of the area of the rectangular area.

When the area of a device marker 4 is detected from the captured image 7 a , the marker detection section 32 sends the area information of the device marker 4 to the identification section 33 . Examples of the area information are the X- and Y-coordinates of a predetermined corner of the device marker 4 and the X-direction width and Y-direction width of the device marker 4 . The above processing is also performed on the position marker 5 in the same manner. Although the captured image 7a of FIG. 4A includes two device markers 4a and 4b, the number of device markers 4 included in each captured image 7 may be other than two.

The description of each block of the device detection device 1 will be continued below with reference to FIG.

The identification unit 33 converts the information of the device marker 4 detected by the marker detection unit 32 into the registration number of the device 43 . In this way, the identifying section 33 can identify the registration number indicated by the device marker 4 . The identification unit 33 further converts the information on the position markers 5 detected by the marker detection unit 32 into information on the sections within the building (hereinafter referred to as "section information"). In this way, the identifying section 33 can identify the section information indicated by the position marker 5 . Examples of section information are numbers or character strings representing each section within a building, and numbers or character strings representing boundaries between sections within a building.

The identification unit 33 acquires area information of the device marker 4 from the marker detection unit 32 . Examples of the area information are the X- and Y-coordinates of the corners of the device marker 4 and the X-direction width and Y-direction width of the device marker 4 . The identification unit 33 extracts the internal area of the device marker 4 (the area within the outer frame of the device marker 4 ) from the captured image 7 based on the area information of the device marker 4 . Specifically, the identification unit 33 calculates the X- and Y-coordinates of the corners of the internal region of the device marker 4 and the X-direction width and Y-direction width of the internal region of the device marker 4 . These values are expressed in pixels, for example. The rate of excluding the outer frame from the device marker 4 is pre-stored in the storage unit 37 as the outer frame exclusion parameter.

The identification unit 33 converts the information of the device marker 4 into the registration number of the device 43 based on the information of the internal area of the device marker 4 . This conversion is performed using a color conversion table stored in the storage unit 37 .

The identification unit 33 may perform image processing for uniforming the luminance distribution in the captured image 7 or image processing for reducing the number of colors in the captured image 7 in order to suppress misidentification. For example, a histogram averaging process or a histogram extension process may be performed for uniforming the luminance distribution in the captured image 7 . In addition, the K-means method may be used to suppress erroneous coloring of cells. The above processing is also performed on the position marker 5 in the same manner.

The collation unit 34 collates the registration number of the device marker 4 identified by the identification unit 33 with the registration number in the database 23 and extracts the registration number corresponding to the registration number of the device marker 4 from the database 23 . Specifically, when the same registration number as the registration number of the device marker 4 exists in the database 23, the matching unit 34 checks the registration number of the device marker 4 (that is, the registration number extracted from the database 23). It is sent to the position estimator 35 . On the other hand, if the same registration number as the registration number of the device marker 4 does not exist in the database 23, the matching unit 34 determines that the detection of the device marker 4 is erroneous, and excludes this detection result from the processing target. do. The registration number of each device 43 is registered in advance in the database 23 . In this embodiment, when a device marker 4 having a certain registration number is created, the registration number may be registered in the database 23 .

The collation unit 34 further collates the division information of the position marker 5 identified by the identification unit 33 with division information in the database 23 to extract division information corresponding to the division information of the position marker 5 from the database 23 . Specifically, when the same section information as the section information of the position marker 5 exists in the database 23, the matching unit 34 compares the section information of the position marker 5 (that is, the section information extracted from the database 23). It is sent to the position estimator 35 . On the other hand, if the same section information as the section information of the position marker 5 does not exist in the database 23, the matching unit 34 determines that the detection of the position marker 5 was erroneous, and excludes this detection result from the processing target. do. In the database 23, block information of a plurality of positions within the building is registered in advance. In this embodiment, when the position marker 5 having certain section information is created, the section information may be registered in the database 23 .

As described above, the position markers 5 are marks used to identify positions within the building, and are provided at multiple locations within the building. The position marker 5 of this embodiment holds section information as information that uniquely indicates the position within the building. In this embodiment, position markers 5 are placed at locations that indicate boundaries between compartments, and multiple position markers 5 are placed at various locations within the building such that each captured image 7 includes at least one position marker 5. It is In addition, the position marker 5 of the present embodiment holds section information in the form of numbers, which are hereinafter referred to as section numbers.

The position marker 5 may be created with the same specifications as the equipment marker 4 or with different specifications, but the position marker 5 of the present embodiment is a color marker created with the same specifications as the equipment marker 4. there is The position marker 5 of this embodiment is composed of a plurality of cells and an outer frame surrounding these cells, and the outer frame is filled with a color that can be separated from the floor 41 . The position marker 5 holds section information that uniquely indicates the position within the building by the color of the cells that constitute the position marker 5 .

The position estimation unit 35 determines (estimates) the installation position of each device 43 by combining the detection result of the device marker 4 and the detection result of the position marker 5 . Specifically, when one device marker 4 and one position marker 5 are detected from one and the same captured image 7, the device 43 is specified based on the registration number indicated by the device marker 4, and the position marker 5 The position of this device 43 is specified based on the section information indicated by . The position estimation unit 35 of this embodiment acquires the Y coordinate of the upper left corner of the device marker 4 and the Y coordinate of the upper left corner of the position marker 5 included in the captured image 7, and compares the values of these Y coordinates. Then, position information indicating the position of the device 43 corresponding to the device marker 4 is determined (set) based on the comparison result. The positional information of this embodiment is determined based on the section information indicated by the positional marker 5 .

FIG. 5 is a diagram showing an example of the captured image 7 output from the imaging device 3 of the first embodiment.

FIG. 5(a) shows a captured image 7b output from one of the imaging devices 3 as an example of the captured image 7. FIG. The captured image 7b includes one device 43 (device 43c) placed on the floor 41, one device marker 4 (device marker 4c) attached to the upper part of this device 43, and the and one position marker 5 (position marker 5a) affixed on the passageway 42. As shown in FIG. FIG. 5(a) further shows the Y coordinate Y1 of the upper left corner of the equipment marker 4c and the Y coordinate Y2 of the upper left corner of the position marker 5a.

FIG. 5B shows a captured image 7c output from one of the imaging devices 3 as an example of the captured image 7. FIG. The captured image 7c includes one device 43 (device 43d) placed on the floor 41, one device marker 4 (device marker 4d) stuck on the upper part of this device 43, and one device marker 4d provided on the floor 41. and two position markers 5 (position markers 5b, 5c) affixed on the passageway 42. As shown in FIG. FIG. 5(b) further shows the Y coordinate Y3 of the upper left corner of the device marker 4d, the Y coordinate Y4 of the upper left corner of the position marker 5b, and the Y coordinate Y5 of the upper left corner of the position marker 5c. .

In the case of FIG. 5A, the marker detection section 32, identification section 33, and collation section 44 operate as follows. The marker detection unit 32 calculates the X- and Y-coordinates of the upper left corner of the position marker 5a and the width in the X direction and the width in the Y direction of the position marker 5a from the captured image 7b of FIG. 5(a). The identification unit 33 calculates the section number of the position marker 5a detected by the marker detection unit 32. FIG. The collation unit 34 collates the division number of the position marker 5a calculated by the identification unit 33 with the division number in the database 23, and extracts from the database 23 the division number corresponding to the division number of the position marker 5a. The above also applies to the case of FIG. 5(b).

The position estimation unit 35 changes the processing for determining position information according to the number of position markers 5 detected from one captured image 7 by the marker detection unit 32 . For example, the captured image 7b in FIG. 5(a) includes one position marker 5a, and position information is determined based on this position marker 5a. On the other hand, the captured image 7b in FIG. 5(b) includes two position markers 5b and 5c, and the position information is determined based on these position markers 5b and 5c. However, when counting the number of position markers 5, only the position markers 5 for which section information has been extracted from the database 23 by the collating section 34 are counted, and the section information has not been extracted from the database 23 by the collating section 34. The position marker 5 is excluded from counting. Determination of the position information is performed using the position markers 5 from which the segment information has been extracted.

As shown in FIG. 5A, when the number of position markers 5 detected by the marker detection unit 32 is one (position marker 5a), the Y coordinate Y1 of the upper left corner of the device marker 4c and the position marker Compare with the Y coordinate Y2 of the upper left corner of 5a. Then, when Y1 is greater than Y2, the section information indicating section P1 is determined as the position information indicating the position of the device marker 4c (device 43c) (Y1>Y2). On the other hand, when Y1 is smaller than Y2 or equal to Y2, the section information indicating section P2 is determined as the position information indicating the position of the device marker 4c (Y1≦Y2).

Section P1 is a section on the +Y direction side of the position marker 5a, and section P2 is a section on the -Y direction side of the position marker 5a. Since Y1 is smaller than Y2 in FIG. 5(a), the section information indicating the section P2 is determined as the position information indicating the position of the device marker 4c. The determined partition information is sent to the registration unit 36 together with the registration number of the device marker 4c.

As shown in FIG. 5B, when the number of position markers 5 detected by the marker detection unit 32 is two (position markers 5b and 5c), the Y coordinate Y3 of the upper left corner of the device marker 4d, The Y coordinate Y4 of the upper left corner of the position marker 5b is compared with the Y coordinate Y5 of the upper left corner of the position marker 5c. Then, when Y3 is larger than Y4, section information indicating section P3 is determined as position information indicating the position of device marker 4d (device 43d) (Y3>Y4). If Y3 is smaller than or equal to Y4 and larger than Y5, section information indicating section P4 is determined as position information indicating the position of device marker 4d (Y5<Y3≦Y4). On the other hand, when Y3 is smaller than Y5 or equal to Y2, section information indicating section P5 is determined as position information indicating the position of device marker 4d (Y3≦Y5).

Section P3 is a section on the +Y direction side of the position marker 5b, section P4 is a section on the -Y direction side of the position marker 5b and on the +Y direction side of the position marker 5c, and section P5 is a section on the -Y direction side of the position marker 5c. This is a section on the Y direction side. In FIG. 5B, since Y3 is smaller than Y4 and larger than Y5, the section information indicating section P4 is determined as the position information indicating the position of device marker 4d. The determined partition information is sent to the registration unit 36 together with the registration number of the device marker 4d.

When the number of position markers 5 detected by the marker detection unit 32 is three or more, the same processing as in the case of two is performed.

The description of each block of the device detection device 1 will be continued below with reference to FIG.

The registration unit 36 registers the registration number and section information of each device marker 4 sent from the position estimation unit 35 in the database 23 . For example, when the registration number "N" and the section information "I" of a certain device marker 4 are registered, the device 43 having the registration number "N" is located in the section indicated by the section information "I". means. The registration number “N” of the device marker 4 and the section information “I” are registered in the database 23 in association with each other as information of the same device 43 . Further, the registration unit 36 may register in the database 23 the time when the registration number and section information of each device marker 4 are obtained from the position estimation unit 35 . Also, the registration unit 36 may register these pieces of information in a location other than the database 23 .

The storage unit 37 has various information necessary for the operation of the device detection device 1 , and has, for example, a position coordinate table indicating the installation position (coordinates) of each position marker 5 . The position coordinate table of this embodiment is created in advance in consideration of the structure of the building and stored in the storage unit 37 .

Note that the information acquired from the storage unit 37 in this embodiment may be acquired from a storage location other than the storage unit 37 instead. Also, the information stored in the storage unit 37 in this embodiment may be stored in a storage location other than the storage unit 37 instead. Examples of such a storage location are the device information management device 2 and a storage device outside the device detection system.

FIG. 6 is a diagram showing an example of data stored in the database 23 of the first embodiment.

FIG. 6 shows, as items of stored data, the registration number (ID) of the device marker 4, the section information indicating the position of the device marker 4, and the timing (update date) when the registration number and the section information were registered in the database 23. , indicates the customer of the product to be manufactured. FIG. 6 shows, as an example, the IDs, section information, update dates, and customers of four devices 43 .

In this embodiment, the status of waiting for shipment may be registered in the database 23 . Also, if the update date of the section information of a certain device 43 is earlier than a predetermined date, the progress delay status of that device 43 may be registered in the database 23 . Furthermore, the registration number of the device marker 4 registered in the database 23 by the registration unit 36 may be the registration number identified by the identification unit 33, or the registration number extracted from the database 23 by the collation unit 34. Any number obtained by converting one of these registration numbers into another number may be used as long as the device marker 4 can be specified.

FIG. 7 is a flowchart for explaining the device detection method of the first embodiment. The device detection method shown in FIG. 7 is performed by the device detection device 1 of this embodiment.

First, the imaging unit 31 acquires the captured image 7 from one of the imaging devices 3 in the building (step S1). The imaging device 3 may continue imaging all day long, or may start imaging at regular intervals and end imaging after a certain period of time. In the latter case, the storage unit 37 stores a list of shooting start times and shooting end times.

Next, the marker detection unit 32 extracts color information from the captured image 7 in order to detect the device marker 4 and the position marker 5 from the captured image 7 (step S2). When the area of the device marker 4 is detected from the captured image 7 (step S3 YES), the process proceeds to step S4. When the area of the device marker 4 is not detected from the captured image 7 (step S3 NO), the process returns to step S1.

Next, the identification unit 33 converts the color information detected by the marker detection unit 32 into the registration number of the device 43 (device marker 4) (step S4). The identification section 33 further converts the color information detected by the marker detection section 32 into the section number of the position marker 5 .

Next, the collation unit 34 collates the registration number obtained by the identification unit 33 with the registration number stored (registered) for collation in the database 23 . If the registration number obtained by the identification unit 33 is registered in the database 23 (step S5 YES), the process proceeds to step S6. If the registration number obtained by the identification unit 33 is not registered in the database 23 (step S5 NO), the process returns to step S1. The collation unit 34 performs the same processing on the block number obtained by the identification unit 33 as well.

Next, the position estimator 35 estimates the position information of the device marker 4 based on the registration number and section number determined to be registered in the database 23 (step S6). Specifically, the position estimating unit 35 determines the section where the device marker 4 is located based on the section number of the position marker 5, and sets the section indicating the section where the device marker 4 is located as the position information of the device marker 4. Determine information. For example, partition information is determined by the method shown in FIGS. 5(a) and 5(b).

Next, the registration section 36 transmits the section information of the device marker 4 determined by the position estimation section 35 to the device information management device 2 together with the registration number of this device marker 4 (step S7). As a result, the registration number and section information of the device marker 4 are registered in the database 23 .

FIG. 8 is a flowchart for explaining the details of the device detection method of the first embodiment. FIG. 8 shows the processing performed on the position marker 5 by the device detection device 1 of this embodiment. In this description, reference is made to FIGS. 5(a) and 5(b).

First, the marker detection unit 32 detects the device marker 4 and the position marker 5 from the captured image 7 (step S11). Next, the identification unit 33 identifies the registration number indicated by the device marker 4 and the section number indicated by the position marker 5 (step S12). Next, after collating these registration numbers and section numbers with the registration numbers and section numbers in the database 23, the process proceeds to step S13.

When the number of position markers 5 detected by the marker detection unit 32 is one (position marker 5a) as in the case where the captured image 7b in FIG. 5A is captured (step S13 YES), position estimation The unit 35 compares the Y coordinate Y1 of the upper left corner of the device marker 4c with the Y coordinate Y2 of the upper left corner of the position marker 5a. Then, when Y1 is greater than Y2 (step S14 YES), the section information indicating the section P1 is determined as position information indicating the position of the device marker 4c (device 43c), and is transmitted to the device information management device 2. (Step S21). On the other hand, if Y1 is smaller than Y2 or equal to Y2 (step S14 NO), section information indicating section P2 is determined and transmitted as position information indicating the position of the device marker 4c (step S22).

On the other hand, when the number of position markers 5 detected by the marker detection unit 32 is two or more (position markers 5b and 5c; two here), as in the case where the captured image 7c in FIG. 5B is captured. (step S13 NO), the position estimation unit 35 calculates the Y coordinate Y3 of the upper left corner of the device marker 4d, the Y coordinate Y4 of the upper left corner of the position marker 5b, and the Y coordinate of the upper left corner of the position marker 5c. Compare with Y5. Then, when Y3 is greater than Y4 (step S15 YES), the section information indicating the section P3 is determined as the position information indicating the position of the device marker 4d (device 43d) and transmitted to the device information management device 2. (Step S23). When Y3 is smaller than Y4 or equal to Y4 and larger than Y5 (step S15 NO, step S16 YES), section information indicating section P4 is determined and transmitted as position information indicating the position of device marker 4d. (step S24). On the other hand, when Y3 is smaller than Y5 or equal to Y2 (step S16 NO), section information indicating section P5 is determined and transmitted as position information indicating the position of device marker 4d (step S25).

As described above, the device detection system of the present embodiment detects the device marker 4 installed on each device 43, and estimates the position in the building where the device marker 4 is located. 43 can be automatically detected. Further, the device detection system of this embodiment can automatically manage the position of the device 43 by registering the detected installation position of the device 43 on the device management system.

(1) First Modification FIG. 9 is a diagram showing the configuration of a device detection system according to a first modification of the first embodiment.

FIG. 9 is a perspective view showing the overall configuration of the device detection system of this modification, showing the wall surface 45 of the building in addition to the components shown in FIG. The position marker 5 in FIG. 3 is provided on the passage 42 installed in the center of the floor 41, but the position marker 5 in FIG. ing. FIG. 9 shows a plurality of position markers 5 provided at high positions on a wall surface 45 extending in the y-direction. The position marker 5 of this modified example may be written on the wall surface 45 with paint or the like, or may be provided on the wall surface 45 in such a manner that a plate or sheet on which the position marker 5 is written is attached to the wall surface. .

The imaging device 3 of this modified example includes imaging devices 3 a to 3 c attached to the crane 6 at an angle facing the floor 41 and an imaging device 3 d attached to the crane 6 at an angle facing the wall surface 45 . The imaging devices 3 a to 3 d move inside the building together with the crane 6 . The device detection device 1 of this modification detects the device marker 4 from the captured images 7 captured by the imaging devices 3a to 3c, and detects the position marker 5 from the captured image 7 captured by the imaging device 3d.

Examples of the captured image 7 captured by the imaging device 3 are, as described above, the captured image 7a in FIG. 4(a), the captured image 7b in FIG. 5(a), the captured image 7c in FIG. As will be described later, there are a captured image 7d in FIG. 10A, a captured image 7e in FIG. 10B, and the like. These figures show the X direction (X coordinate) corresponding to the x direction inside the building, the Y direction (Y coordinate) corresponding to the y direction inside the building, and the Z direction (Z coordinate) corresponding to the z direction inside the building. ). The device detection system of this modified example manages the positions of the device marker 4 and the position marker 5 using X coordinates, Y coordinates, and Z coordinates.

Further details of this modified example will be described below. The configuration of the device detection system shown in FIG. 1 and the configuration of the device detection device 1 shown in FIG. 2 are also applied to this modification.

FIG. 10 is a diagram showing an example of the captured image 7 output from the imaging device 3 of the first modified example of the first embodiment.

FIG. 10(a) shows a captured image 7d captured by the imaging device 3d as an example of the captured image 7. FIG. The captured image 7d includes three position markers 5 (position markers 5d to 5f). FIG. 10A shows the Y direction (Y coordinate) corresponding to the y direction inside the building and the Z direction (Z coordinate) corresponding to the z direction inside the building. FIG. 10(a) further shows the Y coordinate Ya of the position marker 5f and the width Wa of the captured image 7d in the Y direction. The device detection system of this modified example manages the positions of the device marker 4 and the position marker 5 using X, Y, and Z coordinates.

Each of the position markers 5d and 5e holds section information in the building by two-digit numbers "4-3" and "4-4". For example, the position marker 5e is made up of a board on which the number "4-4" is printed, and the number "4-4" is section information indicating one section within the building. The position markers 5d and 5e may hold section information represented by numbers other than two digits, or may hold section information represented by characters or shapes other than numbers. However, it is desirable that the position markers 5d and 5e hold section information that indicates the order in which the sections are arranged in order of numerical size.

The position markers 5f are provided to indicate boundaries between compartments in the building, and are configured by, for example, boards on which triangles are printed. Note that this position marker 5f, unlike the other position markers 5 described above, does not carry an identifier of the position within the building. In this modification, a plurality of first position markers 5 holding identifiers like the position markers 5d and 5e are provided in a row on the wall surface 45, and a plurality of second position markers 5 not holding identifiers like the position markers 5f are provided. A marker 5 is provided between the first position markers 5 . Each of the second position markers 5 of this modified example is composed of a plate on which triangles are printed, but these triangles may be replaced with other figures (for example, circles or straight lines). Also, the second position marker 5 may hold an identifier like the first position marker 5 does.

FIG. 10(b) shows, as an example of the captured image 7, a captured image 7e captured by the imaging device 3a. The captured image 7e is an image captured at the same time as the captured image 7d. The captured image 7 e includes one device 43 (device 43 e ) placed on the floor 41 and one device marker 4 (device marker 4 e ) pasted on the device 43 . FIG. 10(b) further shows the Y-coordinate Y6 of the upper left corner of the equipment marker 4e and the Y-coordinates Y7 and Y8 of the boundaries between the compartments in the building.

In this modification, when the marker detection unit 32 detects the position markers 5d to 5f from the captured image 7d, the identification unit 33 identifies the section information of the position markers 5d and 5e from the position markers 5d and 5e, and the matching unit 34 , position markers 5 d and 5 e are compared with the block information in the database 23 . Then, when the partition information identified from the position markers 5d and 5e exists in the database 23, the position estimation unit 35 determines that the position marker 5f is the position marker 5d (the first position marker holding the partition information 4-3). 5) and the position marker 5e (first position marker 5 holding section information 4-4). Thereby, the position estimation unit 35 can specify the Y coordinate Ya of the position marker 5f based on the information of the position markers 5d and 5e.

Note that instead of determining that the position marker 5f is detected when both the position markers 5d and 5e are detected, the position estimation unit 35 determines that one of the position markers 5d and 5e (for example, the position marker 5e) is detected. In this case, it may be determined that the position marker 5f has been detected. Actually, the position marker 5d in the captured image 7d of FIG. 10A is not detected by the marker detection unit 32 because only half of the position marker 5d is included in the captured image 7d. Therefore, in the following description, it is assumed that, when the position marker 5e is detected, the position estimation unit 35 determines that the position marker 5f adjacent to the position marker 5e in the -Y direction of the position marker 5e has been detected.

On the other hand, the storage unit 37 holds in advance a position coordinate table showing the correspondence between each second position marker 5 and the Y coordinate of the boundary line between sections. The position coordinate table indicates, for example, that the position marker 5f having the Y coordinate Ya corresponds to the Y coordinates Y7 and Y8 of the boundary lines between the sections. Therefore, when the position marker 5f is detected from the captured image 7d, the position estimating unit 35 determines that there is a boundary line having Y coordinates Y7 and Y8 in the captured image 7e captured at the same time as the captured image 7d. judge. As a result, the position estimation unit 35 of this modified example can perform the same processing as the processing for the captured image 7c in FIG. 5(b).

Specifically, the position estimation unit 35 compares the Y-coordinate Y6 of the upper left corner of the device marker 4e, the Y-coordinate Y7 of the boundary line between sections, and the Y-coordinate Y8 of the boundary line between sections. Then, when Y6 is larger than Y7, the section information indicating section P6 is determined as the position information indicating the position of the device marker 4e (device 43e) (Y6>Y7). Also, when Y6 is smaller than or equal to Y7 and larger than Y8, section information indicating section P7 is determined as position information indicating the position of device marker 4e (Y8<Y6≦Y7). On the other hand, when Y6 is smaller than or equal to Y8, the section information indicating section P8 is determined as the position information indicating the position of the device marker 4e (Y6≤Y8).

Section P6 is a section on the +Y direction side of the boundary line having Y coordinate Y7, and section P7 is a section on the -Y direction side of the boundary line having Y coordinate Y7 and on the +Y direction side of the boundary line having Y coordinate Y8. , and the partition P8 is the partition on the −Y direction side of the boundary line having the Y coordinate Y8. Since Y6 is smaller than Y7 and larger than Y8 in FIG. 10B, the section information indicating section P7 is determined as the position information indicating the position of the device marker 4e. The determined partition information is sent to the registration unit 36 together with the registration number of the device marker 4e.

FIG. 11 is a diagram showing an example of the position coordinate table of the first modified example of the first embodiment.

The storage unit 37 of this modified example holds in advance a position coordinate table showing the correspondence relationship between the second position markers 5 such as the position marker 5f and the Y coordinates of the boundary lines between the sections. The position coordinate table indicates, for example, that the position marker 5f having the Y coordinate Ya corresponds to the Y coordinates Y7 and Y8 of the boundary lines between the sections. Note that in the position coordinate table of FIG. 11, the Y coordinate Ya of the position marker 5f is displayed in a unit system different from that of the Y coordinates Y7 and Y8 of the boundary line. The Y-coordinate Ya of the position marker 5f in this modified example indicates the center Y-coordinate of the position marker 5f.

FIG. 12 is a flowchart for explaining the device detection method of the first modification of the first embodiment. FIG. 12 shows the processing performed on the position marker 5 by the device detection device 1 of this modified example. In this description, reference is made to FIGS. 10(a) and 10(b).

First, the marker detection unit 32 detects the position marker 5e from the captured image 7d of FIG. 10(a) (step S31). Next, the identifying section 33 identifies the section information indicated by the position marker 5e (step S32). Next, after collating the section information identified from the position marker 5e with the section information in the database 23, the process proceeds to step S33.

Next, the marker detection unit 32 detects the position marker 5f from the captured image 7d of FIG. 10(b) (step S33). The processing of step S33 is performed by image processing such as template matching processing and pattern recognition processing.

Next, the position estimation unit 35 receives information on the position markers 5e and 5f and reads the position coordinate table from the storage unit 37 (step S34). Accordingly, the position estimation unit 35 can determine that the position marker 5f is the second position marker 5 adjacent to the position marker 5e (the first position marker 5 holding the section information 4-4). Also, the position estimation unit 35 can specify the Y coordinate Ya of the position marker 5f based on the information of the position marker 5e. The position estimation unit 35 can further determine that a boundary line having Y coordinates Y7 and Y8 exists in the captured image 7e captured at the same time as the captured image 7d.

Next, the position estimating unit 35 compares the Y coordinate Y6 of the upper left corner of the device marker 4e in the captured image 7e, the Y coordinate Y7 of the boundary line between sections, and the Y coordinate Y8 of the boundary line between sections. do. Then, when Y6 is greater than Y7 (step S35 YES), the section information indicating section P6 is determined as the position information indicating the position of the device marker 4e (device 43e), and is transmitted to the device information management device 2. (Step S41). When Y6 is smaller than Y7 or equal to Y7 and larger than Y8 (step S35 NO, step S36 YES), section information indicating section P7 is determined and transmitted as position information indicating the position of device marker 4e. (step S42). On the other hand, when Y6 is smaller than or equal to Y8 (step S36 NO), section information indicating section P8 is determined and transmitted as position information indicating the position of the device marker 4e (step S43).

Thus, the device detection system of this modified example includes the position markers 5 provided on the wall surface 45 instead of the floor 41 . As a result, it is possible to prevent the position marker 5 from being stepped on by a person walking in the building (for example, a worker who transports goods), thereby preventing the position marker 5 from being stepped on by a person and deteriorating. .

(2) Second Modification A second modification of the first embodiment will be described with reference to FIG. The configuration of the device detection system shown in FIG. 1 and the configuration of the device detection device 1 shown in FIG. 2 are also applied to this modification.

In FIG. 3, position markers 5 are utilized to estimate the position of each instrument marker 4 . On the other hand, in this modified example, in order to estimate the position of each device marker 4, a characteristic part existing inside the building is used. Examples of such parts are the corners of columns in the building, the color of the floor 41 in the building, and the like. Detection of such a portion is performed, for example, by image processing such as template matching processing and pattern recognition processing.

For example, when using the corner of the pillar to estimate the position of each device marker 4 , the mark detection unit 32 detects the corner of the pillar from the captured image 7 . In this case, the Y coordinates of the corners of the pillars are stored in the storage unit 37 in advance. Then, when the device marker 4 and the corner of the pillar are detected from the captured image 7, the position estimation unit 35 determines the position information of the device marker 4 using the Y coordinate of the corner of the pillar. This position information can be determined by performing the same processing as the processing for the captured image 7b in FIG. 5(a).

In this way, the device detection system of this modified example estimates the position of each device marker 4 by using characteristic parts in the building instead of the position markers 5 . As a result, it becomes possible to avoid the labor required for installation and maintenance of the position marker 5 .

In addition, in this modification, in order to estimate the position of the device marker 4, a plurality of characteristic parts existing in the building may be used. Examples of such parts are the corners of columns in a building. In this case, the difference in shape and color between the pillars may be used in the same way as the difference in section information between the position markers 5 . This makes it possible to determine that a first column is located in a first section of the building, a second column is located in a second section of the building, and so on. In this case, the identification unit 33 identifies the pillars, and the matching unit 34 determines whether the identification of the pillars is correct.

(3) Third Modification A third modification of the first embodiment will be described with reference to FIG. The configuration of the device detection system shown in FIG. 1 and the configuration of the device detection device 1 shown in FIG. 2 are also applied to this modification.

In FIG. 3 , the imaging device 3 that images the position marker 5 is a visible camera, and the position information of the equipment marker 4 is determined using the position marker 5 imaged by the imaging device 3 . On the other hand, in this modified example, another device attached to the crane 6 together with the imaging device 3 is used to determine the position information of the equipment marker 4 . Examples of such devices are laser sensors and stereo cameras.

When using a laser sensor, the laser sensor is mounted on the crane 6 so as to face, for example, the +y direction or the -y direction. The laser sensor moves in ±y directions inside the building together with the imaging device 3 and the crane 6 . When the laser sensor is attached to the crane 6 so as to face the +y direction, the laser sensor measures the distance to the wall of the building located in the +y direction of the crane 6 to determine the position of the imaging device 3. presume. On the other hand, when the laser sensor is attached to the crane 6 so as to face the -y direction, the laser sensor measures the distance to the wall of the building located in the -y direction of the crane 6, and the imaging device Estimate the position of 3. The distance between the wall and the laser sensor can be measured by known methods, for example.

In this case, when the device marker 4 is detected from the captured image 3, the position estimation unit 35 obtains the position information of the device marker 4 from the above-described position information measured by the laser sensor when the captured image 3 was captured. Make a decision based on distance. Specifically, the position of the imaging device 3 is estimated based on the distance measured by the laser sensor, and the position information of the device marker 4 is determined based on the estimated position of the imaging device 3 .

When using a stereo camera, the stereo camera is mounted on the crane 6 so as to face the +y direction or the -y direction, for example. The stereo camera moves in ±y directions inside the building together with the imaging device 3 and the crane 6 . When the stereo camera is attached to the crane 6 so as to face the +y direction, the stereo camera measures the distance to the wall of the building located in the +y direction of the crane 6, thereby determining the position of the imaging device 3. presume. On the other hand, when the stereo camera is attached to the crane 6 so as to face the -y direction, the stereo camera measures the distance to the wall of the building located in the -y direction of the crane 6, and the imaging device Estimate the position of 3. The distance between the wall and the stereo camera can be measured by a known method of calculating the distance in the depth direction, for example by the principle of triangulation. The position information of the instrument marker 4 in this case can be determined in the same way as in the laser sensor.

In this way, the device detection system of this modified example uses a laser sensor or a stereo camera instead of the imaging device 3 to estimate the position of each device marker 4 . As a result, it becomes possible to avoid the labor required for installation and maintenance of the position marker 5 . Note that the laser sensor and stereo camera of this modification may measure the distance between the imaging device 3 and a location other than the wall in the building.

As described above, the device detection apparatus 1 of this embodiment acquires the captured image 7 from the imaging device 3 moving in the building, and detects the device 43 in the building based on the device marker 4 detected from the captured image 7. Identifies and determines the location of the equipment 43 (equipment marker 4) based on information from the imaging device 3 or a device that moves with the imaging device 3 . For example, the device detection apparatus 1 of the present embodiment determines the position of the device 43 (device marker 4) based on the position marker 5 detected from the captured image 7, as described with reference to FIG. .

Therefore, according to the present embodiment, even if the imaging device 3 is not installed at a distance close to the device 43, the device 43 installed in a wide range can be detected by moving the imaging device 3 so as to approach the device 43. It becomes possible to Thus, according to this embodiment, it is possible to efficiently detect the devices 43 installed in a wide range.

Although the imaging device 3 is moved by the crane 6 in this embodiment, it may be moved by other means. For example, the imaging device 3 may be attached to a forklift that moves within the building, or may be attached to the helmet of a worker that moves within the building. This also applies to a second embodiment and a third embodiment, which will be described later.

(Second embodiment)
FIG. 13 is a block diagram showing the configuration of the device detection device 1 of the second embodiment. The device detection device 1 of this embodiment includes a movement detection unit 38 in addition to the components shown in FIG. This embodiment will be described with reference to FIG. Note that the configuration of the device detection system shown in FIG. 1 is also applied to this embodiment.

In the case of the first embodiment, the imaging device 3 may continue imaging all day long, or may start imaging at regular time intervals and end imaging after a certain period of time has elapsed. In the latter case, the storage unit 37 stores a list of shooting start times and shooting end times. The imaging unit 31 acquires the captured image 7 from such an imaging device 3 .

In the case of this embodiment, the movement detection unit 38 detects whether or not the imaging device 3 is moving based on the captured image 7 from the imaging device 3 . Then, the movement detection unit 38 controls the timing at which the imaging unit 31 acquires the captured image 7 from the imaging device 3 for detecting the device marker 4 based on the detection result as to whether the imaging device 3 is moving. do. For example, when it is detected that the imaging device 3 is not moving, the movement detection unit 38 transmits a signal requesting not to acquire the captured image 7 from the imaging device 3 to the imaging unit 31 . On the other hand, when it is detected that the imaging device 3 is moving, the movement detection unit 38 transmits a signal requesting acquisition of the captured image 7 from the imaging device 3 to the imaging unit 31 . The imaging unit 31 determines whether or not to acquire the captured image 7 from the imaging device 3 according to such a signal. As a result, the device detection device 1 of the present embodiment can perform automatic control to cause the imaging unit 31 to acquire the captured image 7 for detecting the device marker 4 only when the imaging device 3 is moving. .

According to this embodiment, the movement detection unit 38 detects movement of the imaging device 3 through image processing, so that it is possible to automatically set the timing for starting detection processing of the device marker 4 and the position marker 5. Become. According to the present embodiment, it is possible to perform such detection processing only when the imaging device 3 is moving. 7 can be saved.

FIG. 14 is a flowchart for explaining the device detection method of the second embodiment. The flowchart of FIG. 14 includes step S8 before steps S1 to S7 shown in FIG.

In step S<b>8 , the movement detection unit 38 determines whether or not the imaging device 3 has moved based on the captured image 7 acquired from the imaging device 3 . Specifically, the movement detection unit 38 calculates the amount of time-series change in the plurality of captured images 7 . The amount of change in time series can be calculated by, for example, obtaining a difference image by comparing each captured image 7 with the captured image 7 of the previous frame or the subsequent frame by background subtraction processing or inter-frame subtraction processing. Also, the amount of change in time series may be calculated by calculating the movement of each pixel by optical flow processing, for example. Alternatively, the movement detection unit 38 compares the captured image and the template image by template matching processing using the captured image 7 before the movement of the imaging device 3 as a template image, thereby determining whether the imaging device 3 has moved. can be judged.

If it is determined that the imaging device 3 has not moved (step S8 NO), the process of step S8 is performed again later. On the other hand, if it is determined that the imaging device 3 has moved (step S8 YES), the process proceeds to step S1. In step S<b>1 , the imaging unit 31 acquires the captured image 7 for detection of the device marker 4 from the imaging device 3 .

As described above, the device detection device 1 of this embodiment includes the movement detection unit 38 that detects movement of the imaging device 3 . Therefore, according to the present embodiment, it is possible to reduce the number of captured images 7 to be processed for detection of the device marker 4 and the like, and to reduce the storage amount of the captured images 7 .

(Third embodiment)
FIG. 15 is a block diagram showing the configuration of the device detection device 1 of the third embodiment. The device detection apparatus 1 of this embodiment includes an image linking unit 39 in addition to the components shown in FIG. This embodiment will be described with reference to FIGS. 3 and 16. FIG. FIG. 16 is a diagram showing an example of a connected image according to the third embodiment. Note that the configuration of the device detection system shown in FIG. 1 is also applied to this embodiment.

The image connecting unit 39 connects a plurality of captured images 7 captured by one or more imaging devices 3, and represents the state of the entire installation location of the device 43 obtained by connecting these captured images 7. Output a concatenated image. Examples of such a connected image are the connected image 8a in FIG. 16(a) and the connected image 8b in FIG. 16(b). A connected image 8a in FIG. 16A is obtained, for example, by connecting a plurality of captured images 7 captured by the imaging devices 3a to 3c in FIG. A connected image 8b in FIG. 16B is obtained, for example, by connecting a plurality of captured images 7 captured by the imaging devices 3a to 3d in FIG. Hereinafter, when describing an arbitrary connected image using these connected images 8a and 8b as an example, this will be referred to as "connected image 8". According to this embodiment, by outputting the connected image 8 that provides a bird's-eye view of the entire building, it is possible to display the position of each device 43 and the positional relationship between the devices 43 in an easy-to-understand manner.

The image connecting unit 39 receives a plurality of captured images 7 from the imaging unit 31 and connects these captured images 7 based on the information received from the position estimating unit 35 . For example, the image linking unit 39 receives positional information of a plurality of device markers 4 included in these captured images 7, and the positional relationship between the device markers 4 in the generated linked image 8 is indicated by this positional information. These captured images 7 are concatenated so as to match the position of the instrument marker 4 . Information other than the position information of the device marker 4 may be used to generate the connected image 8 . Examples of such information are the Y coordinates of the boundaries between compartments, the compartment information of the position markers 5, and the like.

The image linking unit 39 may link the captured images 7 by other methods. For example, the image connecting unit 39 may connect the captured images by superimposing similar regions of the captured images 7 by template matching processing. Further, the image connecting unit 39 calculates a feature amount in the captured images 7 such as an ORB feature amount, and combines the captured images 7 by superimposing similar regions of the captured images 7 based on the feature amount. good too.

Note that the connected image 8a in FIG. 16A is displayed in such a manner that the X axis is vertically oriented and the Y axis is horizontally oriented, that is, the original orientation of each captured image 7 is rotated by 90 degrees. , the connected image 8a may be displayed without such rotation. When displaying the connected image 8a on the display, the connected image 8a may be rotated and displayed, or may be enlarged or reduced for display. This also applies to the connected image 8b in FIG. 16(b).

Here, the connected image 8b in FIG. 16(b) will be described.

FIG. 16(b) shows additional information 9 added to the connected image 8b. The additional information 9 is information about a certain device 43 in the connected image 8b, and is added near the position where this device 43 is arranged in the connected image 8b. The additional information 9 in FIG. 16(b) includes the registration number of this device 43 and the section information. The image linking unit 39 determines the position where the additional information 9 is added in the linked image 8b based on the section information of the device 43. FIG. For example, the connected image 8b is output in a form in which the additional information 9 is added to the connected image 8b, and displayed on the display in a form in which the additional information 9 is superimposed on the connected image 8b.

The connected image 8b of FIG. 16(b) further includes a plurality of position markers 5 provided on the wall surface 45 (see FIG. 9). These position markers 5 may be the position markers 5 actually included in the captured image 7, or may be additions added to the connected image 8b like the additional information 9. FIG.

Further, the additional information 9 may be information other than the registration number of the device 43 and the section information. For example, the additional information 9 may be device information (information about the device 43 ) stored in the database 23 or device information stored in advance in the storage unit 37 . The display color for displaying the additional information 9 may be any color. For example, the additional information 9 may be displayed in a specific color such as white that can be separated from the background, or may be displayed with some transparency so that an image hidden under the additional information 9 can be seen. .

FIG. 17 is a flowchart for explaining the device detection method of the third embodiment. In the flowchart of FIG. 17, step S7 of the flowchart of FIG. 7 is replaced with steps S9 and S10.

In the flowchart of FIG. 17, after the position information is estimated in step S6, the image connecting unit 39 connects the multiple captured images 3 based on the information received from the position estimating unit 35 to create a connected image 8. (Step S9).

Next, the registration unit 36 sends the section information of the device marker 4 determined by the position estimation unit 35 to the device information management apparatus 2 together with the registration number of the device marker 4 and the connected image 8 created by the image connection unit 39 . (step S7). As a result, the registration number and section information of the device marker 4 and the connected image 8 are registered in the database 23 .

As described above, the device detection device 1 of this embodiment includes the image linking unit 39 that links a plurality of captured images 7 captured by one or more imaging devices 3 and outputs the linked image 8. . Therefore, according to the present embodiment, by outputting the connected image 8 that overlooks a wide range in the building, it is possible to display the position of each device 43 and the positional relationship between the devices 43 in an easy-to-understand manner.

The captured images 3 connected by the image connecting unit 39 may be the captured images 3 captured at different times by the same imaging device 3, or the captured images 3 captured at the same time by different imaging devices 3. Alternatively, the captured images 3 captured by different imaging devices 3 at different times may be used.

Now, the program 14 in the device detection device 1 of FIG. 1 will be described.

This program 14 is a file in an installable format or an executable format, and is recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk). provided. Alternatively, the program 14 may be stored on a computer connected to a network such as the Internet, and provided by being downloaded via the network. Also, the program 14 may be provided or distributed via a network such as the Internet. In the first to third embodiments, any method may be used to provide the program 14 in the device detection device 1 . Further, the program 14 may be pre-installed in the ROM within the device detection device 1 .

Although several embodiments have been described above, these embodiments are presented by way of example only and are not intended to limit the scope of the invention. The novel apparatus and methods described herein can be embodied in many other forms. In addition, various omissions, substitutions, and alterations may be made to the forms of the apparatus and methods described herein without departing from the spirit of the invention. The appended claims and their equivalents are intended to cover such forms and modifications as fall within the scope and spirit of the invention.

1: device detection device, 2: device information management device,
3, 3a, 3b, 3c, 3d: imaging device,
4, 4a, 4b, 4c, 4d, 4e: instrument markers,
5, 5a, 5b, 5c, 5d, 5e, 5f: position markers;
6: crane, 7, 7a, 7b, 7c, 7d, 7e: captured image,
8, 8a, 8b: connected image, 9: additional information,
11: Input/output device, 12: Arithmetic device, 13: Storage device, 14: Program,
21: Arithmetic device, 22: Storage device, 23: Database,
31: imaging unit, 32: marker detection unit, 33: identification unit,
34: Verification unit, 35: Position estimation unit, 36: Registration unit,
37: storage unit, 38: movement detection unit, 39: image connection unit,
41: Floor, 42: Passage, 43, 43a, 43b, 43c, 43d, 43e: Equipment,
44: vehicle, 45: wall surface

Claims (11)

an imaging unit that acquires a captured image captured by an imaging device that moves within a building;
From the captured image, a device identification graphic, which is a graphic provided at or near the device for identifying the device in the building, and a graphic provided in the building for identifying the position in the building. a detection unit for detecting a certain position identification figure;
The first identification information obtained from the device identification graphic is collated with the second identification information stored in the storage unit as the identification information of the device, and the second identification information corresponding to the first identification information is obtained. 2 identification information is extracted from the storage unit, and the third identification information obtained from the position identification graphic, and the fourth identification stored in the storage unit as the identification information of the location within the building an extraction unit that extracts the fourth identification information corresponding to the third identification information from the storage unit by collating the information with the information;
When the second identification information corresponding to the first identification information is extracted, using information output from the imaging device or information output from a device that moves with the imaging device, the a determination unit that determines location information indicating the location of the device in the building;
a registration unit that associates and registers the identification information of the device and the location information determined by the determination unit;
with
The device detecting device, wherein the determination unit determines the position information using the position identification graphic in which the fourth identification information corresponding to the third identification information is extracted from the storage unit. 2. The device according to claim 1, wherein the determination unit determines the position information indicating the position of the device by using the position identification graphic detected from the same captured image as the device identification graphic of the device. detection device. 3. The device detection device according to claim 1, wherein said position identification graphic is installed on the floor of said building. 2. The equipment detection device according to claim 1, wherein said position identification graphic is installed on a wall surface of said building. The detection unit detects a predetermined portion within the building from the captured image,
2. The device detection apparatus according to claim 1, wherein said determination unit determines said position information using said predetermined portion detected from said captured image. 2. The device detection device according to claim 1, wherein said determination unit determines said position information using a distance to a predetermined location in said building measured by a device that moves together with said imaging device. 7. The imaging device according to any one of claims 1 to 6, wherein the imaging device is attached to a moving body that moves the imaging device, or to a moving body that moves the imaging device and a device that moves together with the imaging device. equipment detection device. 8. The equipment detection apparatus according to claim 1, wherein said position information is information relating to a section in said building where said equipment is located. further comprising a movement detection unit that uses the captured image to detect whether the imaging device is moving;
The movement detection unit controls the timing at which the imaging unit acquires the captured image from the imaging device for detecting the device identification graphic, using a detection result as to whether or not the imaging device is moving. 9. The equipment detection device according to any one of claims 1 to 8. Image connection for connecting a plurality of the captured images captured by the imaging device using the position information determined by the determining unit, and outputting a connected image obtained by connecting the plurality of captured images. 10. The equipment detection apparatus of any one of claims 1-9, further comprising a unit. 11. The device detection apparatus according to claim 10, wherein said image linking unit adds information of said device to the vicinity of the position of said device in said linked image and outputs said linked image.

Images