JP7801906B2 - Method for detecting the pouring status of cement mixture - Google Patents

Description

The present invention relates to a method for detecting the pouring status of a cement mixture being poured into a pouring area surrounded by formwork, and a program executed to detect the pouring status of the cement mixture.

When constructing a concrete structure, fresh concrete (also known as "ready-mixed concrete") is poured (casting) into an area surrounded by formwork designed and arranged to fit the shape of the concrete structure to be constructed. After that, once the poured fresh concrete has hydrated and developed a certain level of strength, the formwork is removed (de-molding).

Fresh concrete is often produced not at the pouring site but at a fresh concrete manufacturing plant (hereafter referred to as a "ready-mixed concrete plant"). For this reason, taking into account transportation time, there is a time lag of several tens of minutes to over an hour between production and use. At the pouring stage of fresh concrete, this time lag is taken into account and fresh concrete is usually ordered in a quantity slightly larger than expected. In such cases, the unused remaining fresh concrete (hereafter referred to as "remaining concrete") is often discarded as it cannot be stored. Furthermore, some of the remaining concrete is returned to the ready-mixed concrete plant, and how to dispose of it has long been a problem.

There are some factories where the amount of fresh concrete returned to ready-mix concrete plants (hereinafter referred to as "returned concrete") exceeds 100 ^m3 per month, and nationwide it is said to reach 1.5 to 2 million ^m3 per year. For this reason, the existence of returned concrete is an issue not only from the perspective of expenses for ready-mix concrete companies but also from the perspective of the environment in the industry.

One way to reduce leftover and returned concrete is to determine the current pouring height and the fill rate of ready-mixed concrete within the planned pouring area. If the current pouring height and the fill rate of ready-mixed concrete within the planned pouring area can be determined, the true amount of ready-mixed concrete required until the pouring work is complete can be determined, and the true amount of ready-mixed concrete can be ordered based on this detection result.

Technologies described in Patent Documents 1 to 3 below have been proposed as methods for determining the current pouring height and the filling rate of ready-mixed concrete within the planned pouring area.

Japanese Patent Application Publication No. 6-240868 Japanese Patent Application Laid-Open No. 2020-98139 JP 2018-84131 A

Patent Document 1 discloses a method for detecting the pouring surface of concrete using an optical sensor. The optical sensor is used by attaching it to formwork, for example. However, with this method, although the optical sensor is embedded in the concrete after pouring is complete, part of it is exposed on the concrete surface. Furthermore, traces of the cable used to send and receive information remain on the concrete surface. From this perspective, this is undesirable in terms of the aesthetic appearance of the resulting concrete structure.

Patent Document 2 discloses a method of installing an ultrasonic sensor on the outer surface of a formwork to detect the concrete pouring surface. With this method, the ultrasonic sensor can be removed after pouring is complete, so the sensor is not embedded in the concrete and the issues encountered in Patent Document 1 are unlikely to arise. However, this method has issues such as the need to place multiple ultrasonic sensors on the formwork in advance and limiting the concrete pouring target to steel pipes.

Patent Document 3 discloses a method for detecting the height position of the pouring surface using a rotary laser distance sensor. Unlike Patent Document 2, this method does not require the prior installation of multiple sensors on the formwork. However, with this method, the positions that can be measured by the sensor are limited to a straight line, which can result in extremely low accuracy when detecting the pouring height within a planned pouring area where height variations in the depth direction may occur. Furthermore, the laser distance sensor must be installed so that it is facing strictly vertically downward, which makes installation difficult.

The present invention aims to provide a method and an execution program for a processing device that can accurately detect the pouring status of a cement mixture within a planned pouring area while simplifying on-site work.

The present invention is a method for detecting the pouring status of a cement mixture in a pouring area surrounded by a formwork,
A step (a) of virtually arranging a plurality of detection virtual points within the planned pouring area according to a predetermined rule by calculation processing;
A step (b) of imaging a reference marker that is distinguishable from the formwork and the cement mixture and a portion of the planned pouring area on an image captured by the depth camera from a depth camera installed above the upper end of the formwork and overlooking the planned pouring area;
A step (c) of determining, by calculation, pixel virtual points that are the detection virtual points corresponding to each pixel on the captured image based on the relative relationship between the position of the reference marker on the captured image captured in the step (b) and the position of each pixel corresponding to the planned pouring area on the captured image;
The method is characterized by having a step (d) of comparing a value determined by the actual distance between the pixel virtual point obtained based on the captured image taken in step (b) and the depth camera with a value determined by a calculated distance, which is a calculated distance between the detection virtual point corresponding to the pixel virtual point and the depth camera, and detecting the pouring status of the cement mixture based on the comparison result.

A different method to those described in Patent Documents 1 to 3 for detecting the current pouring status is to use a camera to take a picture from above the formwork facing vertically downward, and then detect the current pouring status based on the image obtained. However, with this method, the camera needs to be installed with the imaging direction facing vertically downward so that the entire area to be poured is included in the imaging range.

However, at actual construction sites, it may not be possible to install the camera so that the entire planned pouring area fits within its angle of view. Furthermore, if the entire planned pouring area is to fit within its angle of view, it may be necessary to install the camera at a position significantly higher than the bottom of the planned pouring area. However, because objects that are further away appear smaller to the camera, it may not be possible to accurately detect the height position of the pouring surface within the planned pouring area based on the captured image.

Furthermore, if the camera's imaging direction cannot be set strictly vertically downward, it is necessary to detect how the formwork is viewed from the camera's imaging direction and then perform a correction process on the image. However, the method for this correction process is not clear even when referring to the above-mentioned Patent Documents 1 to 3. Furthermore, this correction process requires information on the camera's tilt angle, and the camera's installation direction must be precisely adjusted on-site. This means that the work may require skill and take a significant amount of time.

In contrast, with the above method, multiple virtual points (detection virtual points) are virtually arranged (set) in the planned pouring area in advance by calculation processing according to a prescribed rule. Then, based on the reference marker displayed in the image captured by the depth camera and information about the formwork within the planned pouring area, the correspondence between each pixel on the captured image and the detection virtual points is determined by calculation processing. The installation angle of the depth camera can be detected by calculation processing based on the positional relationship between the reference marker and the planned pouring area (particularly part of the formwork) on the captured image. In other words, there is no need to precisely adjust the installation angle of the depth camera on site.

In particular, with the method of the present invention, for pixels on the captured image that correspond to the planned pouring area, a pixel virtual point is determined based on the relative positional relationship with the reference marker. In other words, pixels captured at positions outside the planned pouring area are excluded from the process of determining the pixel virtual point.

Since the virtual detection points are set in advance by calculation, if the coordinate position where the depth camera is installed is known, the distance between each virtual detection point and the installation position of the depth camera can be calculated by calculation. In this specification, the distance between each virtual detection point and the installation position of the depth camera calculated by calculation is referred to as the "calculated distance." The installation height of the depth camera can be, for example, a value obtained by measuring it manually or with an optical sensor after installation. If the depth camera is equipped with a sensor, a value automatically detected by the depth camera can also be used.

On the other hand, the coordinate position of the depth camera in directions other than the vertical direction can be calculated from the position of the reference marker on the captured image by calculation. From the perspective of reducing the number of calculations, it is preferable to install the reference marker in a position that is substantially vertically below the depth camera. Note that if the angle between the line virtually connecting the reference marker and the depth camera and the line virtually extending vertically downward from the depth camera is 1° or less, the reference marker can be considered to be installed in a position that is "substantially" vertically below the depth camera. However, in the present invention, the installation position of the reference marker does not necessarily have to be substantially vertically below the depth camera.

The depth camera has the function of recognizing the orientation of each pixel in the captured image relative to the camera's forward direction. In other words, it is possible to recognize the orientation of the imaging point corresponding to each pixel in the captured image captured in step (b) relative to the depth camera.

On the other hand, the virtual detection points virtually arranged in step (a) are points set by computational processing, and therefore their respective coordinate positions are known. As described above, the coordinate position of the depth camera can be calculated by computational processing from the installation height and the position of the reference marker on the captured image. In other words, for example, computational processing based on a geometric method can be used to calculate the direction in which each virtual detection point faces relative to the depth camera.

Then, for each pixel on the captured image, the orientation of the imaging point corresponding to the planned pouring area relative to the depth camera is compared with the orientation of the virtual detection point relative to the depth camera, and the virtual detection point whose orientation is closest to the orientation of the imaging point corresponding to pixel i1 located within the planned pouring area is selected. Specifically, if the orientation of virtual detection point j1 relative to the depth camera is closest to the orientation of the imaging point corresponding to pixel i1 located within the planned pouring area relative to the depth camera, virtual detection point j1 is recognized as the point corresponding to pixel i on the captured image. Similarly, a process is performed to determine virtual detection point j2 corresponding to another pixel i2. In this specification, virtual detection points associated with each pixel on the captured image are referred to as "virtual pixel points."

The depth camera has the function of measuring the distance between the camera (more specifically, the image sensor) and each pixel in the captured image. In other words, the depth camera measures the distance between each pixel associated with a virtual detection point, i.e., each virtual pixel point, and the depth camera. In this specification, the distance between each virtual pixel point and the installation position of the depth camera, measured by the depth camera's function, is referred to as the "measured distance."

On the other hand, pixel virtual points are detection virtual points associated with pixels on the captured image, so as described above, the "calculated distance" between each detection virtual point and the depth camera has already been calculated.

The value determined by the calculated distance at each virtual pixel point calculated in this way is compared with the value determined by the measured distance in step (d). In the simplest method, the calculated distance is compared with the measured distance in step (d).

For example, if the measured distance is greater than the calculated distance, it means that the depth camera is capturing an image at or further from the location of the virtual detection point, i.e., farther away than the depth camera. In this case, it is estimated that no cement mixture is present at the location of the virtual detection point.

On the other hand, if the measured distance is less than the calculated distance, it means that the depth camera is capturing an image closer to the depth camera than the location of the virtual point for detection. In this case, it is estimated that a cement mixture is present at the position of the virtual point for detection.

In other words, by comparing the calculated distance with the measured distance for each pixel virtual point, it is possible to estimate whether or not a cement mixture is present at the location of the detection virtual point corresponding to the pixel virtual point.

At actual construction sites, it is often difficult to place a depth camera directly above the formwork due to the risk of interference with the concrete pump outlet. In such cases, the depth camera is installed not directly above the formwork, but from above outside the formwork, overlooking the planned pouring area diagonally downward. However, with the above method, even when the depth camera is installed in such a position overlooking the planned pouring area diagonally downward, the orientation of each detection virtual point relative to the depth camera can be calculated based on the pixel position of the reference marker displayed in the captured image. Therefore, the detection virtual point closest to the orientation of each pixel relative to the depth camera can be selected, and the pouring status of the cement mixture can be detected by comparing the calculated distance with the measured distance for each selected detection virtual point (i.e., pixel virtual point).

Depending on the pouring site, it is conceivable that the depth camera can be installed vertically above the planned pouring area. Even in this case, the pouring status of the cement mixture can be detected using the above method. In other words, the present invention does not exclude the case where, in step (b), an image of the reference marker and part of the planned pouring area is captured from a depth camera installed vertically above the planned pouring area.

As mentioned above, with this method, pixel virtual points are determined for pixels on the captured image that represent the planned pouring area. Therefore, even if the imaging range of the depth camera extends outside the planned pouring area, erroneous detection of the filling status of the cement mixture based on information from pixels outside the planned pouring area can be avoided.

When the imaging range of the depth camera is part of the planned pouring area, the filling status of the cement mixture estimated based on information about pixels on the captured image that overlap the planned pouring area is, strictly speaking, limited to the planned pouring area that overlaps the imaging range. However, the height of the pouring surface of the cement mixture can be considered to be roughly flat. In other words, if the filling rate of the cement mixture in the planned pouring area that overlaps the imaging range is calculated, it is possible to directly estimate that filling rate as an approximation of the overall filling rate of the planned pouring area.

Step (c) may include a step of determining a closed area including a continuous line obtained by successively selecting pixels corresponding to at least two sides of the upper end of the formwork on the captured image, and determining, through calculation processing, the pixel virtual points corresponding to the pixels on the captured image that are included within the closed area.

In the above example, we have simply compared the magnitude relationship between the actual measured distance and the calculated distance, but it is also possible to compare a value obtained by calculation processing based on the actual measured distance with a value obtained by calculation processing based on the calculated distance. As an example, from the perspective of making even small differences more apparent, it is also possible to compare a value obtained by multiplying the calculated distance by a predetermined gain coefficient with a value obtained by multiplying the actual measured distance by a predetermined gain coefficient.

The above-described method eliminates the need to precisely adjust the installation angle of the depth camera, and the processes of locating the detection virtual points, determining the pixel virtual points based on the image information from the depth camera, and determining whether or not the cement mixture has been poured at the position of the virtual point corresponding to the pixel virtual point are all performed by calculation. This simplifies work at the construction site. Furthermore, the depth camera does not necessarily need to be installed vertically downward, and the pouring status within the planned pouring area can be detected while taking into account the imaging direction of the depth camera.

In this specification, the term "cement mixture" to be poured is a concept that encompasses both fresh concrete and fresh mortar.

The type of fiducial marker is not important, as long as it can be distinguished in an image captured by a depth camera from the formwork and cement mixture reflected in the surrounding area. The fiducial marker can be an object independent of the formwork, or it can be a mark painted on part of the formwork or scaffolding. In the former case, it can be a pole of a different color (e.g., red, yellow-green, etc.) from the concrete, iron pipes, and formwork that may be present in the surrounding area, or a plate with a specific pattern (e.g., an AR marker) that can be recognized by image analysis on the camera side.

The step (c)
a step (c1) of converting, based on a relative relationship between the position of the reference marker and the position of each pixel on the captured image, the coordinate positions of the plurality of virtual detection points on a horizontal-vertical coordinate system formed by a vertical axis parallel to the vertical direction and two axes perpendicular to the vertical axis, into coordinate positions on a camera coordinate system formed by an imaging optical axis parallel to the imaging direction of the depth camera and two axes perpendicular to the imaging optical axis;
A step (c2) of calculating, by calculation processing, an orientation relative to the depth camera, which is defined by two tilt angles relative to the imaging optical axis, based on information about the coordinate positions of the plurality of virtual points for detection on the camera coordinate system obtained in the step (c1);
The method may further include a step (c3) of determining, as the virtual pixel point, a pixel on the captured image that is closest to the orientation relative to the depth camera obtained in the step (c2).

In step (c3), if there are multiple virtual detection points corresponding to pixels on the captured image, all of the virtual detection points may be determined as pixel virtual points. In this case, in step (d), for each pixel virtual point, a value determined by the measured distance from the depth camera may be compared with a value determined by the calculated distance, and the pouring status of the cement mixture may be detected based on the results of these comparisons.

More specifically, assume that three points j1, j2, and j3 are extracted as virtual detection points (virtual pixel points) corresponding to pixel i1. In this case, for each virtual pixel point (j1, j2, j3), the value determined by the measured distance is compared with the value determined by the calculated distance.

Step (d) can be a step of calculating the current filling rate of the cement mixture within the planned pouring area.

For example, the step (a) is a step of virtually arranging a plurality of the detection virtual points on the bottom surface of the planned pouring area and between the bottom surface and a position of the planned pouring height in a vertical direction,
In step (d), among the plurality of detection virtual points, the number of filled-corresponding virtual points that are determined to be filled with the cement mixture and the number of non-filled-corresponding virtual points that are determined to be not filled with the cement mixture may be detected, and the filling rate of the cement mixture in the planned pouring area may be calculated based on the ratio of the filled-corresponding virtual points to the total number of the filled-corresponding virtual points and the non-filled-corresponding virtual points.

As described above, it is also possible to estimate whether a cement mixture is present or absent at the location of a detection virtual point corresponding to a pixel virtual point. Therefore, as an alternative method, in step (d), a point from each detection virtual point that is closest to the boundary between the presence/absence of a cement mixture is determined, and the surface connecting these boundary points is estimated to be the pouring surface. This allows the height of the pouring surface at the current time to be estimated, and it is also possible to calculate the filling rate of the cement mixture based on the ratio of the estimated pouring height to the planned pouring height. The estimated pouring height may also be calculated by averaging the heights of each point that is close to the boundary between the presence/absence of a cement mixture.

Based on the information on the filling rate of the cement mixture obtained using the above method, it is possible to estimate the amount of additional cement mixture required until pouring is complete. Therefore, by placing additional orders with a ready-mix concrete plant, for example, based on this estimate, it is possible to reduce leftover concrete and returned concrete.

Steps (b), (c), and (d) may be repeatedly executed at a predetermined timing. For example, the predetermined timing may be when one or several transport trucks' worth of fresh concrete are poured, as the fresh concrete is transported intermittently from the ready-mix concrete plant to the pouring site. However, this is not limited to this, and the timing may be set, for example, during pouring, or at any other timing, such as during a user's break from pouring work. Additionally, from the perspective of reducing concrete return, it is better to calculate the filling rate in a timely manner as the filling nears completion. Therefore, the predetermined timing may be set to increase the frequency of execution of steps (b), (c), and (d) as the filling rate in the formwork increases.

This makes it possible to detect changes over time in the pouring status of the cement mixture during pouring work.

The method for detecting the pouring status of the cement mixture further includes a step (e) of preparing a processing device including a memory unit,
The step (a) is a step in which the arithmetic processing device determines the relative coordinate positions of each detection virtual point with respect to the depth camera based on information about the dimensions of the planned pouring area and information about the installation position of the depth camera, and records the determined coordinate positions in the memory unit;
The step (b) includes a step of transmitting information based on an image captured by the depth camera to the arithmetic processing device;
the step (c) includes a step in which the arithmetic processing device determines the pixel virtual point based on information on each pixel on the captured image acquired in the step (b) and information on each coordinate position of the detection virtual point recorded in the storage unit;
The step (d) may include a step in which the arithmetic processing device calculates the calculated distance based on the coordinate positions of the detection virtual points relative to the installation position of the depth camera recorded in the memory unit, and compares the calculated distance with the actual measured distance based on the captured image acquired in the step (c).

The arithmetic processing device used in the above method can be one or more of the following: a notebook computer, a tablet computer, a smartphone, a headless computer connected to a sensor via wired or wireless communication (e.g., Bluetooth (registered trademark)), and a server. If the arithmetic processing device is a notebook computer, a tablet computer, or a smartphone, the arithmetic processing is performed at the construction site. If the arithmetic processing device is a server, the arithmetic processing can be performed at a location remote from the construction site, and the results can be sent to another terminal (e.g., a smartphone) located at the construction site.

The method for detecting the pouring status of the cement mixture further includes a step (f) of measuring the height position of the installed depth camera from the bottom surface of the planned pouring area and recording the measurement result in the memory unit of the arithmetic processing device,
The step (a) may include a step in which the arithmetic processing device acquires information regarding the installation position of the depth camera based on information regarding the height position of the depth camera recorded in the memory unit.

The present invention is a program for detecting the pouring status of a cement mixture, which is installed in a calculation processing device and executed to detect the pouring status of a cement mixture in a pouring area surrounded by a formwork,
A process of virtually arranging a plurality of detection virtual points within the planned pouring area according to a predetermined rule;
A process of receiving input of an image captured by a depth camera installed above the upper end of the formwork and overlooking the planned pouring area, the image including a reference marker that can be distinguished from the formwork and the cement mixture in the image captured by the depth camera, and a portion of the planned pouring area;
A process of determining pixel virtual points, which are the detection virtual points corresponding to each pixel on the captured image, based on a relative relationship between the position of the reference marker on the input captured image and the position of each pixel corresponding to the planned pouring area on the captured image;
The present invention is characterized in that the arithmetic processing device executes a process of comparing the actual measured distance between the pixel virtual point obtained based on the input captured image and the depth camera with the calculated distance, which is the calculated distance between the detection virtual point corresponding to the pixel virtual point and the depth camera, and detecting the pouring status of the cement mixture based on the comparison result.

The processing device used in the above method may be one or more of the following: a notebook computer, a tablet computer, a smartphone, and a server.

The present invention simplifies work at the pouring site while enabling accurate detection of the pouring status of the cement mixture within the planned pouring area.

1 is a flowchart schematically showing the procedure for executing a method for detecting the pouring status of a cement mixture. This is a diagram showing a schematic diagram of a depth camera installed at a concrete pouring site. This is a diagram that schematically shows the planned pouring area and the area that can be captured by the depth camera before pouring. This is another diagram that schematically shows the planned pouring area and the area that can be captured by the depth camera before pouring. This is a schematic diagram showing a side view of a depth camera capturing an area to be poured, surrounded by formwork. 1 is a diagram showing the relationship between a camera coordinate system and a horizontal and vertical coordinate system. 10 is a diagram schematically illustrating a state in which a plurality of virtual detection points are virtually arranged; 10 is another diagram schematically illustrating a state in which a plurality of virtual detection points are virtually arranged. FIG. 2 is a functional block diagram illustrating a configuration of a processing unit. 10 is a diagram for explaining the coordinate positions of arranged virtual points for detection; 1 is a diagram illustrating the orientation of an arbitrary point set in the space of the camera coordinate system relative to a depth camera. 1 is a diagram schematically illustrating the positional relationship between an imageable area including a reference marker and a depth camera. 10 is a schematic diagram of a captured image in a case where the area outside the formwork is included in the imaging range of the depth camera.

The following describes in detail an embodiment of a method for detecting the pouring status of a cement mixture and a program for detecting the pouring status of a cement mixture according to the present invention, with reference to the drawings.

This method is a simple way to detect the current pouring status, such as the filling rate of a cement mixture, during the pouring of fresh concrete or other cement mixture into a pouring area surrounded by formwork. In addition to fresh concrete, the cement mixture can also be fresh mortar.

Figure 1 is a flowchart that schematically illustrates the steps involved in carrying out the method for detecting the pouring status of a cement mixture according to the present invention. In the following explanation, the step numbers in Figure 1 will be used as appropriate.

(Step S1: Installing the depth camera)
The depth camera is installed at a position where it can capture an image of the planned pouring area. Figure 2 is a diagram that shows a schematic diagram of a depth camera installed at a pouring site. In the example of Figure 2, the depth camera 11 is attached to a support 15 and is positioned so that it can capture an image of the area surrounded by the formwork 2. The support 15 may be a dedicated pillar for installing the depth camera 11, or may be a vertical extension of scaffolding made of iron pipes or the like installed at the site.

The depth camera 11 is not particularly limited in terms of its type, as long as it is an imaging device that can acquire depth information (distance information to the target object) in addition to image information. For example, imaging devices using so-called stereo camera, ToF, or projector methods can be used. Of the methods listed above, projector-type imaging devices are preferred. When using an image sensor such as a stereo camera, it can be difficult to detect feature points for distance measurement when the inside of the formwork is dark. With ToF methods, accuracy tends to decrease as the distance gets shorter, and there is a risk of accuracy decreasing near the end of pouring when the pouring surface becomes higher.

Figure 2 shows a schematic diagram of a predetermined amount of cement mixture 3 already poured within the planned pouring area. In Figure 2, the upper surface 3a of the cement mixture 3 corresponds to the current pouring surface.

Figure 3A is a diagram that schematically illustrates the planned pouring area and the imageable area of the depth camera 11 before pouring. In this embodiment, the bottom surface 5 of the planned pouring area is located within the imageable area 11i of the depth camera 11. For reference, Figure 3A also illustrates the surface at the planned pouring height (pouring height) (hereinafter referred to as the "planned pouring top surface 6").

However, depending on the conditions at the pouring site, it may be necessary to install the depth camera 11 so that the imageable area 11i captures only a portion of the bottom surface 5 of the planned pouring area, as shown in Figure 3B, for example. Furthermore, adjusting the position and orientation of the depth camera 11 so that almost the entire bottom surface 5 of the planned pouring area is included in the imageable area 11i, as shown in Figure 3A, may require a lot of time for the installation work of the depth camera 11. The method for detecting the pouring status of a cement mixture according to the present invention can detect the pouring status of the cement mixture within the planned pouring area even if the position of the imageable area 11i is shifted relative to the planned pouring area, as shown in Figure 3B.

For ease of explanation, we will first explain how to detect the pouring status of the cement mixture when the depth camera 11 is installed so that almost the entire bottom surface 5 of the planned pouring area is located within the imageable area 11i of the depth camera 11, as illustrated in Figure 3A. We will then explain how the pouring status of the cement mixture can be detected even when only a portion of the bottom surface 5 of the planned pouring area is located within the imageable area 11i of the depth camera 11, as illustrated in Figure 3B.

Figure 4 is a schematic diagram showing the depth camera 11 capturing an image of the planned pouring area surrounded by formwork 2, viewed from the side. The imageable area 11i shown in Figures 3A and 3B is determined by the angle of view 11c of the depth camera 11 shown in Figure 4. In the example shown in Figure 3A, the imageable area 11i roughly corresponds to the area surrounded by formwork 2. On the other hand, in the example shown in Figure 3B, the imageable area 11i includes part of the area surrounded by formwork 2, but also includes the area outside the area surrounded by formwork 2.

In the explanations in this specification, two coordinate systems will be referenced as appropriate. Below, these coordinate systems will be referred to as the "camera coordinate system" and the "horizontal and vertical coordinate system."

The camera coordinate system is a coordinate system determined by three axes expressed using capital letters "X," "Y," and "Z." The camera coordinate system is an X-Y-Z coordinate system, with the Z axis being the front direction of the depth camera 11, i.e., the direction parallel to the imaging optical axis (the front direction of the depth camera 11 is the +Z direction), the X axis being the left-right direction relative to the front of the depth camera 11 (the right direction is the +X direction), and the Y axis being the up-down direction (the up direction is the +Y direction).

In this specification, when a direction needs to be expressed as either positive or negative, it is indicated by adding a positive or negative sign, such as "+X direction" and "-X direction." Furthermore, when a direction is expressed without distinguishing between positive and negative, it is simply referred to as "X direction." In other words, in this specification, when simply referring to the "X direction," it includes both the "+X direction" and the "-X direction." The same applies to the Y direction and the Z direction.

The horizontal-vertical coordinate system is a coordinate system determined by three axes expressed using lowercase "x," "y," and "z." It is an x-y-z coordinate system with the vertical direction as the y-axis (the upward direction is the +y direction), the direction perpendicular to the y-axis and the Z-axis of the camera coordinate system as the x-axis, and the direction perpendicular to the y-axis and x-axis as the z-axis. The z-axis is defined as positive or negative so that the direction at which it intersects with the Z-axis of the camera coordinate system at an angle of 90° or less is the +z direction. The positive and negative directions of the x-axis are defined as a left-handed system with respect to the directions of the y-axis and z-axis. The origin O (see Figure 5) of both the camera coordinate system and the horizontal-vertical coordinate system is the center position of the imaging surface of the depth camera 11.

The relationship between the camera coordinate system and the horizontal-vertical coordinate system set in this way is shown in Figure 5. In Figure 5, the angle ψ between the Z axis of the camera coordinate system, which corresponds to the front direction of the depth camera 11, and the y axis of the horizontal-vertical coordinate system, which corresponds to the vertical direction, is referred to below as the "upward angle."

In Figures 2 to 4, the axial directions of the camera coordinate system defined by the X-Y-Z coordinate system and/or the axial directions of the horizontal and vertical coordinate systems defined by the x-y-z coordinate system are shown for reference. This also applies to the following drawings. However, for ease of explanation, in Figures 2 and 4, the x direction is assumed to be parallel to the horizontal plane.

A position directly above the formwork 2 could interfere with the concrete pump outlet. For this reason, as shown in Figures 2 and 4, the depth camera 11 is installed vertically above the top end of the formwork 2, in a position that overlooks the planned pouring area diagonally downward.

(Step S3: Setting of reference markers)
A reference marker 13 is installed within the image capturing area 11i (see FIG. 3A) of the depth camera 11. In FIGS. 2 and 4, an example is shown in which the reference marker 13 is attached to a support 15 on which the depth camera 11 is installed.

The fiducial marker 13 is imaged by the depth camera 11, as described below, and is used to convert the horizontal-vertical coordinate system to the camera coordinate system based on the pixel position information of the fiducial marker 13 displayed in the captured image. At a concrete pouring site, fresh concrete, iron pipes for forming scaffolding, and formwork 2 are typically present. Therefore, it is expected that the concrete, iron pipes, formwork 2, and ground will be reflected in the image captured by the depth camera 11. The fiducial marker 13 is positioned in a manner that allows it to be distinguished from the concrete, iron pipes, formwork 2, and ground in the captured image. Specifically, the fiducial marker 13 can be a pole of a color (e.g., red, yellow-green, etc.) different from the concrete, iron pipes, and formwork, or a plate with a specific pattern (e.g., an AR marker) that can be recognized by image analysis on the camera side.

It is preferable that the reference marker 13 has a certain size so that the center position of the reference marker 13 can be detected on the captured image. For example, it is preferable that the reference marker 13 is large enough to occupy an area of 3 x 3 pixels or more on the captured image. Note that if the reference marker 13 is realized by an AR marker, the size of the reference marker 13 is set so that it occupies an area on the captured image that is equal to or larger than the minimum number of pixels required for AR marker analysis.

In this embodiment, the reference marker 13 is installed vertically below the depth camera 11. Installing it in this manner reduces the amount of calculation required when converting the horizontal-vertical coordinate system to the camera coordinate system. However, in the present invention, the installation position of the reference marker 13 is not limited to a position vertically below the depth camera 11, as long as it is within the imageable area 11i of the depth camera 11.

When installing the reference marker 13, it is also possible to mark the support 15, formwork 2, or part of the scaffolding with paint of a specified color.

(Step S5: Array of virtual points for detection)
By the calculation process, a plurality of virtual detection points are virtually arranged in the planned concrete pouring area according to a predetermined rule. Fig. 6 is a diagram showing a schematic diagram of a state in which a plurality of virtual detection points are virtually arranged.

In many cases, the dimensions of the planned pouring area are known in advance from design drawings, etc. Even if the dimensions of the planned pouring area are unknown, if formwork 2 has already been installed, the dimensions of the planned pouring area in the horizontal direction can be determined by measuring the dimensions of the area enclosed by formwork 2. Note that information about the pouring height (height of the planned pouring top surface 6) is usually known at the start of pouring work.

In step S5, multiple virtual points (detection virtual points 30) are placed on the bottom surface 5 of the planned pouring area at a predetermined interval in the surface direction. Then, multiple detection virtual points 30 are placed at a predetermined interval in the vertical direction from the coordinate position of each detection virtual point 30 placed on the bottom surface 5 to the height position of the planned pouring top surface 6. Note that the placement of these detection virtual points 30 is performed purely through computation, so there is no need for workers to actually place physical markers at the pouring site.

The arrangement of the detection virtual points 30 is not limited. As an example, as shown in FIG. 6, multiple detection virtual points 30 may be arranged on the outer surface of a rectangular parallelepiped or at positions inside the rectangular parallelepiped. As another example, as shown in FIG. 7, multiple detection virtual points 30 may be arranged on the outer surface of a cylindrical shape or at positions inside the cylindrical shape.

Figures 6 and 7 show a case where the detection virtual points 30 are arranged from the height position of the bottom surface 5 of the planned pouring area to the height position of the planned pouring top surface 6. However, the detection virtual points 30 may be arranged from a position lower than the bottom surface 5, or conversely, the detection virtual points 30 may be arranged to a position higher than the height position of the planned pouring top surface 6. In this step S5, the detection virtual points 30 are arranged so that the planned pouring area is included in the space in which the detection virtual points 30 are arranged three-dimensionally.

However, the calculation process performed in step S5 can be a process for determining the relative coordinate positions of each virtual detection point 30 to be placed. For example, a three-dimensional vector of each virtual detection point 30 is determined, with the position of a specific virtual detection point 31 as the reference point.

Steps S9 and S11, which will be described later, are executed by a processing unit. Figure 8 is a functional block diagram that schematically shows the configuration of this processing unit. Information regarding the relative coordinate positions of each detection virtual point 30 obtained by executing step S5 is recorded in the memory unit 22 of the processing unit 20. The memory unit 22 is a recording medium that stores information, such as a non-volatile memory or a hard disk.

This step S5 and steps S9 and S11, which will be described later, may be executed by the same arithmetic processing device 20. In this case, this step S5 is performed by the arithmetic execution unit 21 of the arithmetic processing device 20. The arithmetic execution unit 21 is a functional means for performing arithmetic processing, and is typically a processor capable of executing programs.

In step S5, the installation position of the depth camera 11 can be used as the reference position for determining the relative coordinate position of each detection virtual point 30. In other words, by using a detection virtual point 30 whose relative positional relationship with the installation position of the depth camera 11 is known as a specific detection virtual point 31 used as the reference for relative coordinates, the coordinate position of each detection virtual point 30 relative to the depth camera 11 can be determined. As an example, as shown in FIG. 9, a detection virtual point 30 installed on the bottom surface 5 of the planned pouring area and vertically below the depth camera 11 can be used as the specific detection virtual point 31. In this case, the coordinate position of each detection virtual point 30 relative to the depth camera 11 can be determined based on the values related to the pouring height y6 and the installation height y11 of the depth camera 11 relative to the planned pouring top surface 6.

As described above, the detection virtual point 30, which is installed on the bottom surface 5 of the planned pouring area and vertically below the depth camera 11, is set as the reference detection virtual point 31. As described above, in the horizontal-vertical coordinate system, the installation position of the depth camera 11 is set as the origin O, so the coordinates of the detection virtual point 31 in the horizontal-vertical coordinate system are (x, y, z) = (0, -(y11+y6), 0).

The bottom surface 5 of the planned pouring area corresponds to the plane y = -(y11 + y6) in the horizontal-vertical coordinate system. Multiple detection virtual points 30 are arranged on this plane. For convenience, the multiple detection virtual points 30 arranged on the plane y = -(y11 + y6) are referred to as the "bottom reference point group." As described above with reference to Figures 6 and 7, the bottom reference points may be arranged in a grid pattern or on a circumference. However, the bottom reference point group is arranged so that it is distributed over the entire bottom surface 5 of the planned pouring area, which is located at least inside the formwork 2. In this case, it is acceptable for part of the bottom reference point group to be located outside the planned pouring area.

When n (n is a natural number) bottom reference points are placed, the coordinates of the i-th (i is a natural number less than or equal to n) bottom reference point in the horizontal-vertical coordinate system are defined as (x, y, z) = (x _i , -(y11+y6), z _i ).

The top surface 6 of the planned pouring area corresponds to the plane y = -y11 in the horizontal-vertical coordinate system. This plane corresponds to a position y6 vertically upward, i.e., in the +y direction, from the plane y = -(y11 + y6) that constitutes the bottom surface reference point group. For convenience, the multiple detection virtual points 30 arranged on the plane y = -y11 are referred to as the "pouring surface reference point group." When n pouring surface reference points are arranged, the coordinates of the i-th pouring surface reference point in the horizontal-vertical coordinate system are defined as (x, y, z) = ( _xi , -y11, _zi ).

Next, multiple detection virtual points 30 are placed so as to divide the space between the bottom surface reference point group and the pouring surface reference point group located vertically above them into m divisions (m is a natural number). For example, the coordinates in the horizontal-vertical coordinate system of the jth detection virtual point 30 (m is a natural number equal to or less than j) when counting vertically upward from the i-th bottom surface reference point are defined as (x, y, z) = ( _xi , -(y11 + y6) + (y11 × (j/m)), _zi ). When the detection virtual points 30 are placed in this manner, the total number of placed detection virtual points 30 is n × m.

When the detection virtual points 30 are arranged in this manner, the coordinates of each detection virtual point 30 in the horizontal-vertical coordinate system are identified. As described above, in the horizontal-vertical coordinate system, the installation position of the depth camera 11 is set as the origin O, so the determined coordinate position of each detection virtual point 30 corresponds to the relative coordinate position with respect to the depth camera 11. Note that if the installation position of the depth camera 11 is not the origin, the coordinate information of the determined detection virtual point 30 can be converted into a three-dimensional vector based on the coordinates of the depth camera 11 in the horizontal-vertical coordinate system, and the relative coordinate position of each detection virtual point 30 with respect to the depth camera 11 can similarly be obtained.

Note that steps S1, S3, and S5 may be performed in any order.

(Step S7: Photographing with depth camera)
After steps S1, S3, and S5 are performed, the depth camera 11 captures an image of the planned pouring area surrounded by the formwork 2. As described above with reference to FIG. 3A, in step S3, the reference marker 13 is installed within the imageable area 11i of the depth camera 11. Therefore, in step S7, the depth camera 11 captures an image of the planned pouring area as well as the reference marker 13. Data 11d related to the obtained captured image is sent from the depth camera 11 to the arithmetic processing device 20. When the arithmetic processing device 20 receives the image data 11d via the transmitter/receiver 23, it stores the data in the memory 22 (see FIG. 8). The transmitter/receiver 23 is a communication interface for exchanging information with the depth camera 11. However, the image information of the depth camera 11 may be input to the arithmetic processing device 20 via a recording medium such as an SD card.

In this embodiment, part of the imageable area 11i of the depth camera 11 may include the outside of the formwork 2, as shown in Figure 3B.

(Step S9: Determine virtual pixel point)
Based on the captured image captured by the depth camera 11, the arithmetic processing unit 20 performs a computation process of associating each pixel on the captured image with a virtual detection point 30. More specifically, based on the image data 11d and information related to the coordinate positions of each virtual detection point 30 recorded in the storage unit 22, the computation execution unit 21 performs a process of associating each pixel on the captured image with a virtual detection point 30. In this specification, the virtual detection points 30 associated with each pixel are referred to as "pixel virtual points." Specifically, when performing this process, a process of converting the coordinate positions of each virtual detection point 30 in the horizontal-vertical coordinate system performed in step S5 into coordinate positions in the camera coordinate system is performed. This coordinate conversion method will be described with reference to FIGS. 10 and 11 .

Figure 10 is a diagram illustrating the orientation of an arbitrary point P placed in the space of the camera coordinate system defined by the X-Y-Z coordinate system, relative to the depth camera 11. The orientation relative to the depth camera 11 is defined by the left-right angle θ and the up-down angle φ.

The left-right angle θ is defined as the angle between the Z axis and the line segment connecting the projection point and the origin when the target point is projected onto the XZ plane of the camera coordinate system. Taking point P shown in Figure 10 as an example of the target point, the angle θp formed by the line segment connecting projection point Pxz and origin O when point P is projected onto the XZ plane and the Z axis corresponds to the left-right angle of point P. If the X coordinate of point P is a negative value, the left-right angle θp will be a negative value.

The vertical angle φ is defined as the angle between the line segment connecting the target point and the origin and the line segment connecting the target point and the origin when the target point is projected onto the XZ plane. Taking point P in Figure 10 as an example, the angle φp formed by the line segment connecting point P and the origin O and the line segment connecting projected point Pxz and the origin O corresponds to the vertical angle of point P. However, if the Y coordinate of point P is a negative value, the vertical angle φp will be a negative value.

The following equations (1) to (3) hold true according to geometric rules between the coordinates (X, Y, Z) = (Xp, Yp, Zp) of point P in the camera coordinate system, the left-right angle θp, the up-down angle φp, and the distance Dp from the depth camera 11 to point P.

Conversely, if the orientation of point P relative to the depth camera 11 (left-right angle θp, up-down angle φp) and the distance Dp from the depth camera 11 are known, the coordinates of point P in the camera coordinate system (X, Y, Z) = (Xp, Yp, Zp) can be calculated using the following equations (4) to (6) based on geometric rules.

The depth camera 11 generally has an array of α pixels horizontally and β pixels vertically (α and β are natural numbers). These pixels define a rectangular imageable area 11i. For each pixel, the depth camera 11 determines the direction in which the image is captured relative to the front direction of the depth camera 11, i.e., the horizontal angle θ and vertical angle φ corresponding to the pixel. Furthermore, the depth camera 11 can obtain the distance D between the object being imaged and the depth camera 11 for each pixel.

Therefore, the coordinate position in the camera coordinate system of each pixel shown in the captured image obtained by the depth camera 11 can be determined by calculating the above equations (4) to (6).

Note that the depth camera 11 is typically defined with a field of view of _θc on the left and right and _φc on the up and down sides, centered on the front direction, and is unable to capture an image beyond this field of view. In other words, if the range of the left and right angles to be captured is _-θc ≦ θ≦ _θc and the range of the up and down angles is _-φc ≦ φ≦ _φc , then points in directions that do not satisfy these can be recognized as being outside the imaging range of the depth camera 11.

As described above, the reference marker 13 is placed in step S3. Here, from the perspective of reducing the number of calculations, the reference marker 13 is placed vertically below the depth camera 11. Figure 11 is a diagram that schematically shows the positional relationship between the imageable area 11i, which includes the reference marker 10, and the depth camera 11.

When the reference marker 10 is positioned at the center of the left-right direction (X direction) on the captured image, the left-right angle θ of the position of the reference marker 10 relative to the depth camera 11 is θ = 0. Here, as shown in Fig. 11, the up-down angle φ of the pixel capturing the reference marker 13 relative to the depth camera 11 is assumed to be φ = φ _m .

The vertical angle φ _m shown in Fig. 11 is the angle formed by the imaging optical axis of the depth camera 11 with respect to the vertical direction, i.e., coincides with the upward angle ψ described above with reference to Fig. 5. In this way, the reference marker 13 is used when calculating the upward angle ψ from the pixel position displayed on the captured image.

Even if the reference marker 13 is not located vertically below the depth camera 11, the upward angle ψ can be calculated using a geometric calculation based on the relative positional relationship between the installation positions of the depth camera 11 and the reference marker 13, and the left-right angle θ and the up-down angle φ corresponding to the pixel on the captured image showing the reference marker 13.

A method for converting coordinate information in the horizontal-vertical coordinate system of point P in space into coordinate information in the camera coordinate system will be explained with reference to Figure 5. It can be seen from Figure 5 that if the horizontal-vertical coordinate system defined by the x-y-z coordinate system is rotated by -(90-ψ) degrees around the y-axis and z-axis while keeping the x-axis fixed, it will coincide with the camera coordinate system defined by the X-Y-Z coordinate system.

That is, as shown in FIG. 5, if the coordinates of point P in space in the horizontal-vertical coordinate system are (x, y, z) = ( _xp , _yp , _zp ), the coordinates of this point in the camera coordinate system, (X, Y, Z) = ( _Xp , _Yp , _Zp ), are calculated using the following equations (7) to (9).

Conversely, if the coordinates (X, Y, Z) = (Xp, Yp, Zp) of point P in space in the camera coordinate system are known, the coordinates ( _x , y, z) = ( _xp , _yp , _zp ) of this point _P in _the horizontal-vertical coordinate system can be calculated using the following equations (10) to (12).

For ease of explanation, the above description has been given using an example in which the left-right direction in the camera coordinate system of the depth camera 11, i.e., the X-axis, is parallel to the horizontal plane. However, even if the depth camera 11 is tilted in the direction of rotation around the imaging optical axis, i.e., the Z-axis, as the rotation axis, i.e., the X-axis is not parallel to the horizontal plane, it is possible to convert between the camera coordinate system and the horizontal-vertical coordinate system using calculation processing based on a geometric method similar to that described above.

In step S5 described above, the relative coordinates of each virtual detection point 30 with respect to the depth camera 11, i.e., coordinate information in the horizontal-vertical coordinate system, are obtained. Furthermore, in step S9, as described above, the upward angle ψ is calculated based on the pixel position of the reference marker 13 displayed on the captured image. Therefore, in accordance with equations (10) to (12) above, coordinate information in the camera coordinate system for each virtual detection point 30 can be obtained.

Based on the coordinate information of each virtual detection point 30 in the camera coordinate system, more specifically the X and Y coordinates, virtual detection points 30 located within the imageable area 11i of the depth camera 11 are extracted. Calculating the X and Y coordinates of each virtual detection point 30 in the camera coordinate system is equivalent to calculating the orientation of each virtual detection point 30 relative to the depth camera 11.

The pixel at the coordinate position that most closely matches the coordinate position in the camera coordinate system of the extracted virtual detection point 30 is identified. This identified pixel corresponds to the "virtual pixel point" described above. Note that if multiple virtual detection points 30 overlap in the Z direction in the camera coordinate system, these multiple virtual detection points 30 may correspond to the same virtual pixel point.

Next, we will explain the case where the imageable area 11i of the depth camera 11 includes the area outside the formwork 2, as shown in Figure 3B. Figure 12 is a diagram that schematically shows an image captured by the depth camera 11 in the manner shown in Figure 3B.

As shown in Figure 3B, if the imageable area 11i captured by the depth camera 11 includes the outside of the formwork 2, the outside of the formwork 2 will also be reflected in the captured image. In this case, the captured image will often only reflect a portion of the planned pouring area surrounded by the formwork 2.

Within the imageable area 11i, area 11ib, which is outside the formwork 2, is not the area where the cement mixture 3 is to be poured. For this reason, area 11ib needs to be excluded from the calculations involved in the pouring status detection process.

Therefore, in step S9, prior to the process of determining the virtual pixel points, a process is performed to determine the pixels to be used for calculation from among the pixels constituting the captured image. As an example, as shown in FIG. 12, the captured image is analyzed to identify the location of the formwork 2 reflected in the image (typically the location of the upper end of the formwork 2). Then, multiple points 41 located on the formwork 2 are connected in series, and a closed area 11ia is identified, defined by these connected lines and the perimeter of the imageable area 11i. Then, the calculation processing device 20 performs a calculation process using the method described above to associate the multiple pixels present in this closed area 11ia with the virtual detection points 30. That is, in the case of the captured image shown in FIG. 12, the pixels included in area 11ib are excluded from the process of matching them with the virtual detection points 30.

Note that the process of identifying this closed area 11ia may include a process in which an operator manually plots the target points 41 using the arithmetic processing device 20 while checking the captured image. Alternatively, the arithmetic processing device 20 may automatically identify the closed area 11ia based on a comparison of the contrast that appears on the captured image, as shown in FIG. 12.

(Step S11: Detection of pouring status)
As described above, the depth camera 11 obtains the distance D between the captured object and the depth camera 11 for each pixel. Information about this distance D is stored as a captured image or together with the captured image in the storage unit 22. In other words, by reading out the information stored in the storage unit 22, the calculation execution unit 21 can recognize the distance (measured distance) between the depth camera 11 and the location captured by the pixel corresponding to the pixel virtual point identified in step S9.

Furthermore, by executing step S5, the relative coordinate positions of multiple detection virtual points 30 with respect to the depth camera 11 are recorded in the memory unit 22. Therefore, the calculation execution unit 21 can recognize the calculated distance (calculated distance) between the detection virtual point 30 corresponding to the pixel virtual point identified in step S9 and the depth camera 11 by reading the information stored in the memory unit 22 and performing calculation processing based on a geometric method as necessary.

First, for simplicity's sake, it is assumed that each pixel virtual point is associated with a single detection virtual point 30. If the measured distance between each pixel virtual point and the depth camera 11 is greater than the calculated distance between the detection virtual point 30 corresponding to the pixel virtual point and the depth camera 11, this means that the depth camera 11 is capturing an image in the +Z direction relative to the target detection virtual point 30. This means that at the X and Y coordinate positions of the target detection virtual point 30 in the camera coordinate system, no component that reflects imaging light, i.e., the cement mixture 3, is captured between the detection virtual point 30 and the depth camera 11 in the Z direction. In this case, it is estimated that the cement mixture 3 is not filled at the position of the target detection virtual point 30.

On the other hand, if the measured distance between each pixel virtual point and the depth camera 11 is less than the calculated distance between the detection virtual point 30 corresponding to the pixel virtual point and the depth camera 11, this means that the depth camera 11 is capturing a position closer to the depth camera 11 in the -Z direction than the target detection virtual point 30. This means that at the X and Y coordinate positions of the target detection virtual point 30 in the camera coordinate system, the cement mixture 3 is captured between the detection virtual point 30 and the depth camera 11 in the Z direction. In other words, it is estimated that the cement mixture 3 is filled at the position of the target detection virtual point 30.

From a similar perspective, when multiple detection virtual points 30 are associated with one pixel virtual point, if the measured distance between each pixel virtual point and the depth camera 11 is greater than the calculated distance between all of the detection virtual points 30 corresponding to the pixel virtual point and the depth camera 11, the depth camera 11 can estimate that the cement mixture 3 has not been filled at the positions of all of the target detection virtual points 30. Furthermore, if the measured distance between the pixel virtual point and the depth camera 11 is less than or equal to the calculated distance between any of the detection virtual points 30 corresponding to the pixel virtual point and the depth camera 11, it can be estimated that the cement mixture 3 has been filled at the position of that detection virtual point 30.

As a detailed example, the calculation execution unit 21 performs the above estimation process on all detection virtual points 30 associated with pixels in the captured image. Furthermore, the calculation execution unit 21 counts the number of virtual points determined to be filled with the cement mixture 3 (hereinafter referred to as "filled virtual points") and the number of virtual points determined to be not filled with the cement mixture 3 (hereinafter referred to as "non-filled virtual points"). The calculation execution unit 21 then calculates the ratio of filled virtual points to the total number of detection virtual points 30 that are the subject of estimation, i.e., the total number of filled virtual points and non-filled virtual points. Based on this calculation result, the calculation execution unit 21 calculates the filling rate of the cement mixture in the planned pouring area and outputs it to the output unit 24. The output unit 24 is, for example, a display monitor. The calculation result by the calculation execution unit 21 may be transmitted to another operating terminal, such as a smartphone, via the transmission/reception unit 23.

As described above with reference to Figure 12, if pixel virtual points are identified only for pixels located within closed region 11ia, then, strictly speaking, the above method calculates the filling rate for the region within the planned pouring area that corresponds to closed region 11ia. However, the bottom surface 5 within the planned pouring area can essentially be considered to be a nearly horizontal plane. Furthermore, as the pouring of the cement mixture 3 progresses, the height of the top surface 3a of the cement mixture 3 can be considered to be roughly constant, regardless of the coordinate position on the xz plane. In other words, the value of the filling rate within the region corresponding to closed region 11ia, estimated by calculation, can be used to approximate the value of the filling rate for the entire planned pouring area surrounded by formwork 2.

As an alternative method, the calculation execution unit 21 performs the above estimation process on all detection virtual points 30 associated with pixels on the captured image, and then identifies a filled-corresponding virtual point adjacent to the non-filled-corresponding virtual point. The identified filled-corresponding virtual point corresponds to the boundary between the area filled with the cement mixture 3 and the area not filled with it, and is therefore referred to below as a "filled boundary virtual point."

The calculation execution unit 21 estimates the current height of the pouring surface of the cement mixture 3 by calculating the surface connecting each of the identified fill boundary virtual points. As described above, since the value of the planned pouring height (y11 + y6) is known, it is also possible to calculate the filling rate of the cement mixture 3 by calculating the ratio of the estimated pouring surface height to the planned pouring height. Note that if pixel virtual points are identified only for pixels located within the closed region 11ia, the above method estimates the height of the pouring surface of the cement mixture 3 within the region corresponding to the closed region 11ia. However, for the same reason, the height of the pouring surface within the region corresponding to the closed region 11ia can be approximated to the height of the pouring surface in the entire planned pouring region surrounded by the formwork 2, and therefore it is possible to approximate the value of the filling rate in the entire planned pouring region based on the value of this estimated height.

The volume of the planned pouring area or the area of the bottom surface 5 of the planned pouring area is often already known or can be easily determined at the construction site. Therefore, once an estimate of the filling rate of the cement mixture 3 is obtained, the required amount of cement mixture 3 can be easily calculated.

By repeatedly performing steps S7, S9, and S11 at predetermined times, it is possible to detect changes over time in the casting status of the cement mixture 3. However, the present invention does not exclude the case where steps S7, S9, and S11 are performed only once.

In the above embodiment, an example was described in which the depth camera 11 captures an image of the planned pouring area diagonally downward. However, the above technique can also be applied if the depth camera 11 can be installed above the planned pouring area so that it captures an image facing vertically downward. If the depth camera 11 could be installed so that it captures an image facing completely vertically downward, then in theory, there would be no need to convert the coordinate system using the reference marker 13. However, in reality, the task of adjusting the orientation of the depth camera 11 so that the imaging direction faces completely vertically downward is cumbersome. In contrast, the process of converting the coordinate system using the reference marker 13 using the above procedure and then determining the pixel virtual point can be performed by calculation processing by the calculation processing device 20, which has the advantage of eliminating the need to precisely adjust the orientation of the depth camera 11.

2: Formwork 3: Cement mixture 3a: Top surface 5: Bottom surface 6: Top surface to be poured 10: Reference marker 11: Depth camera 11c: Angle of view 11d: Image data 11i: Imageable area 11ia: Closed area 11ib: Area not subject to calculation 13: Reference marker 15: Support 20: Calculation processing device 21: Calculation execution unit 22: Memory unit 23: Transmitting/receiving unit 24: Output unit 30: Virtual point for detection 31: Virtual point for detection 41: Point on formwork 42: Continuous line θp: Left-right angle φp: Up-down angle ψ: Looking-up angle

Claims (10)

A method for detecting the pouring status of a cement mixture in a pouring area surrounded by a formwork,
A step (a) of virtually arranging a plurality of detection virtual points within the planned pouring area according to a predetermined rule by calculation processing;
A step (b) of imaging a reference marker that is distinguishable from the formwork and the cement mixture and a portion of the planned pouring area on an image captured by the depth camera from a depth camera installed above the upper end of the formwork and overlooking the planned pouring area;
A step (c) of determining, by calculation, pixel virtual points that are the detection virtual points corresponding to each pixel on the captured image based on the relative relationship between the position of the reference marker on the captured image captured in the step (b) and the position of each pixel corresponding to the planned pouring area on the captured image;
A method for detecting the pouring status of a cement mixture, characterized by comprising a step (d) of comparing a value determined by the actual distance between the pixel virtual point obtained based on the captured image taken in step (b) and the depth camera with a value determined by a calculated distance, which is the calculated distance between the detection virtual point corresponding to the pixel virtual point and the depth camera, and calculating the filling rate of the cement mixture in the planned pouring area based on the comparison result. The method for detecting the pouring status of a cement mixture according to claim 1, characterized in that step (c) includes a step of determining a closed area including a continuous line obtained by successively selecting pixels corresponding to at least two sides of the upper end of the formwork on the captured image, and determining, through calculation processing, the pixel virtual points corresponding to pixels included in the closed area among the pixels on the captured image. The step (c)
a step (c1) of converting, based on a relative relationship between the position of the reference marker and the position of each pixel on the captured image, the coordinate positions of the plurality of virtual detection points on a horizontal-vertical coordinate system formed by a vertical axis parallel to the vertical direction and two axes perpendicular to the vertical axis, into coordinate positions on a camera coordinate system formed by an imaging optical axis parallel to the imaging direction of the depth camera and two axes perpendicular to the imaging optical axis;
A step (c2) of calculating, by calculation processing, an orientation relative to the depth camera, which is defined by two tilt angles relative to the imaging optical axis, based on information about the coordinate positions of the plurality of virtual points for detection on the camera coordinate system obtained in the step (c1);
A method for detecting the pouring status of a cement mixture as described in claim 1 or 2, characterized in that it includes a step (c3) of determining, as the pixel virtual point, the pixel on the captured image that is closest to the orientation relative to the depth camera obtained in step (c2). The method for detecting the pouring status of a cement mixture according to any one of claims 1 to 3, characterized in that in step (d), if the actual measured distance is greater than the calculated distance, it is determined that the cement mixture has not been filled at the position of the virtual point for detection within the planned pouring area, and if the actual measured distance is equal to or less than the calculated distance, it is determined that the cement mixture has been filled at the position of the virtual point for detection within the planned pouring area. The step (a) is a step of virtually arranging a plurality of the detection virtual points on the bottom surface of the planned pouring area, and virtually arranging a plurality of the detection virtual points at equal intervals between each of the detection virtual points arranged on the bottom surface and a position of the planned pouring height,
5. The method for detecting the pouring status of a cement mixture as described in claim 4, characterized in that in step (d), the number of filled-corresponding virtual points that are determined to be filled with the cement mixture and the number of non-filled-corresponding virtual points that are determined to be not filled with the cement mixture are detected among the plurality of detection virtual points, and the filling rate of the cement mixture in the planned pouring area is calculated based on the ratio of the filled-corresponding virtual points to the total number of the filled-corresponding virtual points and the non-filled-corresponding virtual points. The method for detecting the pouring status of a cement mixture according to any one of claims 1 to 5, characterized in that steps (b), (c), and (d) are repeatedly performed at predetermined times. The method further includes a step (e) of preparing a processing unit including a memory unit;
The step (a) is a step in which the arithmetic processing device determines the relative coordinate positions of each detection virtual point with respect to the depth camera based on information about the dimensions of the planned pouring area and information about the installation position of the depth camera, and records the determined coordinate positions in the memory unit;
The step (b) includes a step of transmitting information based on an image captured by the depth camera to the arithmetic processing device;
The step (c) includes a step in which the arithmetic processing device determines the pixel virtual point based on information on each pixel corresponding to the planned pouring area on the captured image acquired in the step (b) and information on each coordinate position of the detection virtual point recorded in the memory unit,
The method for detecting the pouring status of a cement mixture according to any one of claims 1 to 6, characterized in that step (d) includes a step in which the arithmetic processing device calculates the calculated distance based on the coordinate positions of the detection virtual point relative to the installation position of the depth camera recorded in the memory unit, and compares the calculated distance with the actual measured distance based on the captured image acquired in step (c). Further comprising a step (f) of measuring the height position of the installed depth camera from the bottom surface of the planned pouring area and recording the measurement result in the memory unit of the arithmetic processing device,
The method for detecting the pouring status of a cement mixture as described in claim 7, characterized in that step (a) includes a step in which the arithmetic processing device acquires information regarding the installation position of the depth camera based on information regarding the height position of the depth camera recorded in the memory unit. The method for detecting the pouring status of a cement mixture described in any one of claims 1 to 8, characterized in that the reference marker is installed in a position substantially vertically below the depth camera. The method for detecting the pouring status of a cement mixture described in any one of claims 1 to 9, characterized in that the depth camera is installed in a position that overlooks the intended pouring area diagonally downward.