JP7784697B2 - Analysis system and analysis method - Google Patents

Description

The present invention relates to an analysis system and an analysis method.

Conventionally, there is technology for creating environmental maps using sensors (for example, Patent Document 1).

JP 2019-67001 A

However, conventional technology was only able to grasp the position and shape of objects in space, and was not able to distinguish between people and non-people objects in space.

The present invention aims to provide an analysis system and analysis method that can distinguish between objects in space and non-human objects.

The analysis system of the present invention comprises:
a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis system comprising:
The processing unit
an initial background extraction process for extracting one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registering, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction process for extracting one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction process for extracting one or more movable stationary objects from the one or more moving objects and redefining each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
The device is configured to:

In the analysis system of the present invention,
The processing unit, in the initial background extraction process,
an object extraction process for extracting one or more objects based on the point cloud data output from the range measurement sensor at the measurement time within the predetermined initial background extraction period;
a high-position stationary object extraction process for extracting, as stationary objects, one or more first objects at least a portion of which is located at a height position equal to or higher than a predetermined height from among the one or more objects extracted by the object extraction process, and registering, in the cell array, each cell constituting each of the stationary objects as the initial background;
a linear stationary object extraction process that calculates a first linearity evaluation value representing the degree of linearity of one or more second objects other than the one or more first objects extracted by the high-position stationary object extraction process among the one or more objects extracted by the object extraction process, extracts one or more second objects whose first linearity evaluation value exceeds a predetermined first linearity evaluation value threshold as a stationary object, and registers each cell constituting each of the stationary objects in the cell array as the initial background;
It is preferable that the device is configured to perform the following.

In the analysis system of the present invention,
The processing unit, in the movable stationary object extraction process,
It is preferable that the system is configured to perform an interlocking movable stationary object extraction process in which a first moving object among the one or more moving objects transitions from a sitting state to a standing state, and then a new moving object is extracted around the first moving object by the moving object extraction process, and then, if the time during which the following equation (5) is continuously satisfied exceeds a predetermined duration threshold within a predetermined new moving object monitoring time from the time when the equation (5) is first satisfied, the new moving object is extracted as a movable stationary object, and each cell that constitutes the movable stationary object in the cell array is redefined as a post-defined background.
In the above formula (5),
h(k,t): Height of the new moving object (mm)
γ: A given real number between 0 and 200
h _{sit_ave} (p, q): average sitting height of the first moving object (mm)

In the analysis system of the present invention,
It is preferable that the system is configured to perform a linear movable stationary object extraction process in which, when an occluded area appears in the same location in the initial background for a predetermined occluded area monitoring time, a second moving object among the one or more moving objects that is located between the range sensor and the occluded area is identified, a second linearity evaluation value representing the degree of linearity of the object is calculated for the second moving object, and when the second linearity evaluation value exceeds a predetermined second linearity evaluation value threshold, the second moving object is extracted as a movable stationary object, and each cell that constitutes the movable stationary object in the cell array is redefined as a post-defined background.

In the analysis system of the present invention,
The processing unit, in the movable stationary object extraction process,
It is preferable that the system is configured to perform a highest point slight fluctuation type movable stationary object extraction process in which, when the highest point of a third moving object among the one or more moving objects is located at approximately the same position over a specified highest point monitoring time, the third moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background.

In the analysis system of the present invention,
The processing unit, in the movable stationary object extraction process,
It is preferable that the method is configured to perform a voxel number slight variation type movable stationary object extraction process in which, when a fourth moving object among the one or more moving objects has a voxel number variation evaluation value representing the degree of variation in the number of voxels constituting the fourth moving object within a predetermined voxel number monitoring time period that is less than a predetermined voxel number variation evaluation value threshold, the fourth moving object is extracted as a movable stationary object and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background.

The analysis method of the present invention includes:
a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis method using an analysis system comprising:
an initial background extraction step in which the processing unit extracts one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registers, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction step in which the processing unit extracts one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction step in which the processing unit extracts one or more movable stationary objects from the one or more moving objects, and redefines each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
Includes.

The present invention provides an analysis system and analysis method that can distinguish between humans and non-human objects in a space.

1 is a schematic diagram illustrating an analysis system according to an embodiment of the present invention. 1 is a schematic flowchart of an analysis method according to an embodiment of the present invention. 1 is a diagram for explaining types of objects; 1 is a diagram for explaining an example of a predetermined space to be analyzed and an example of a cell array; 5 is a diagram for explaining analysis data in a state where point cloud data of the predetermined space in FIG. 4 at a certain measurement time within a predetermined initial background extraction period is stored in a cell array. 6 is a diagram for explaining analysis data obtained after performing an initial background extraction process on the analysis data of FIG. 5 . 5 is a diagram for explaining analysis data obtained after performing a moving object extraction process on point cloud data in a predetermined space in FIG. 4 at a certain measurement time within a predetermined observation period. 10 is a diagram for explaining an interlocking movable stationary object extraction process; 10 is a diagram for explaining a linear movable stationary object extraction process; 10 is a diagram for explaining a linear movable stationary object extraction process; 10 is a diagram for explaining a voxel number slight variation type movable stationary object extraction process. 10 is a diagram for explaining a voxel number slight variation type movable stationary object extraction process. 7 is a diagram for explaining analysis data obtained after a movable object extraction process is performed on the analysis data of FIG. 6 . 1 is a diagram illustrating an example of an environmental map.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an analysis system and an analysis method according to an embodiment of the present invention will be described with reference to the accompanying drawings.
In each drawing, the same components are denoted by the same reference numerals.

[Analysis System 1]
First, an analysis system 1 according to one embodiment of the present invention, which can use the analysis method of the present invention, will be described with reference to Figures 1 and 4. Figure 1 shows a schematic configuration of the analysis system 1. The analysis system 1 is configured to analyze a predetermined space S (Figure 4). As shown in Figure 1, the analysis system 1 includes a range sensor 2 and an analysis device 3. Figure 4 shows an overhead view of an example of the predetermined space S.
The predetermined space S to be analyzed by the analysis system 1 is preferably an indoor space such as an office space or a school space.
Specifically, the analysis system 1 is configured to distinguish between humans and non-human objects among objects in a predetermined space S, and analyze how humans use the predetermined space S. Examples of users of the analysis system 1 include analysts, owners, and users of the predetermined space S.

Here, with reference to FIGS. 3 and 4, classification of objects in a predetermined space S will be described. As illustrated in FIG. 4, various objects O (Oa to Og) exist within the predetermined space S. As illustrated in FIG. 3, the objects O can be classified into stationary objects SO and moving objects MO. Stationary objects SO refer to objects that do not move at all during the measurement period of the range sensor 2, and examples thereof include walls, doors, and desks. In the example of FIG. 4, of the objects Oa to Og within the predetermined space S, objects Oa to Oc are stationary objects SO, and of these, objects Oa to Ob are walls, and object Oc is a desk. Moving objects MO refer to objects that move at least temporarily during the measurement period of the range sensor 2. Moving objects MO can be classified into people P and movable stationary objects MSO. Movable stationary objects MSO refer to objects that move due to interaction with people P during the measurement period of the range sensor 2, and examples thereof include chairs, desks, whiteboards, potted plants, and trash cans. 4, of the objects Oa to Og in the predetermined space S, the objects Od to Og are moving objects, and of these, the object Oe is a person P, and the objects Od, Of, and Og are movable stationary objects MSO. Of the objects Od, Of, and Og as movable stationary objects MSO, the object Od is a chair, the object Of is a potted plant, and the object Og is a trash can.
The analysis system 1 is configured to distinguish a person P in a predetermined space S from stationary objects SO and movable stationary objects MSO, which are objects other than a person.

An xyz Cartesian coordinate system is defined in a predetermined space S. As shown in Fig. 4, the x-axis and y-axis are parallel to the horizontal direction, and the z-axis is parallel to the height direction.
As will be described later, the analysis device 3 of the analysis system 1 analyzes a predetermined space S using a two-dimensional cell array CA. As shown in FIG. 4 , the cell array CA is defined by subdividing the predetermined space S into a grid-like structure from a bird's-eye view. The cell array CA is parallel to the xy plane. The cells C of the cell array CA are arranged along the x-axis and the y-axis. Each cell C of the cell array CA forms a square. The length of one side of the cell C is preferably 13 cm or less, and more preferably 11 cm or less, from the viewpoint of preventing multiple objects O from being entirely contained within a single cell C. Furthermore, the length of one side of the cell C is preferably 7 cm or more, and more preferably 9 cm or more, from the viewpoint of preventing an increase in the processing load of the analysis device 3. The length of one side of the cell C is preferably 10 cm, for example.
The coordinates of each cell C are expressed as (pq) in an xy Cartesian coordinate system, where p and q are preferably integers.

(Surveillance sensor 2)
The range sensor 2 is configured to measure the distance to an object O in a predetermined space S at each measurement time every predetermined measurement time interval (e.g., 0.1 seconds) over a predetermined measurement period and output three-dimensional point cloud data. The range sensor 2 is preferably configured, for example, as a 3D-LiDAR (3D Laser Imaging Detection and Ranging) or a 3D-LRF (3D Laser Range Finder) configured to measure the distance to an object O in the predetermined space S every predetermined measurement time interval using a laser and output three-dimensional point cloud data. Each point of the three-dimensional point cloud data is represented by an x-y-z coordinate system.
Because the range sensor 2 is configured to output three-dimensional point cloud data, it can include height information and has fewer blind spots than if it were configured to output two-dimensional point cloud data, enabling more accurate analysis of the position, shape, and number of objects O. Furthermore, range sensors (such as 3D-LiDAR) configured to output three-dimensional point cloud data generally have a wider measurement range than range sensors (such as 2D-LiDAR) configured to output two-dimensional point cloud data, which also has the advantage of allowing the number of range sensors to be reduced.
The number of range sensors 2 is preferably one, but may be multiple. When multiple range sensors 2 are used, the analysis device 3 may use point cloud data obtained by combining point cloud data from the multiple range sensors 2.
The length of the predetermined measurement period is, for example, 0.5 to 5 hours.
The range measurement sensor 2 is installed so that its detection range includes the entire predetermined space S.
The range sensor 2 is preferably installed at a height slightly higher than the person P in the specified space S. For example, the range sensor 2 is preferably installed at a height of 1.75 to 1.90 m from the ground, and more preferably at a height of 1.80 to 1.85 m from the ground.
The point cloud data obtained by the range measurement sensor 2 is output to the analysis device 3. The range measurement sensor 2 transmits the point cloud data to the analysis device 3 in real time via communication (wired communication and/or wireless communication) at each measurement time for each predetermined measurement time interval.

(Analysis device 3)
1, the analysis device 3 includes a processing unit 31, a communication unit 32, a storage unit 33, an input unit 34, and a display unit 35. The analysis device 3 can be configured from any computer, such as a personal computer, a tablet terminal, a mobile terminal, or a dedicated device. The analysis device 3 may be configured from one computer or multiple computers.

The processing unit 31 is composed of, for example, one or more CPUs, and executes programs such as an analysis program stored in the memory unit 33, thereby controlling the entire analysis device 3, including the communication unit 32, memory unit 33, input unit 34, and display unit 35, and performing processes such as the initial background extraction process, moving object extraction process, movable stationary object extraction process, and environmental map creation process, which will be described later. Details of the processes performed by the processing unit 31 will be described later.

The communication unit 32 is configured to communicate (wired communication and/or wireless communication) with the range measurement sensor 2. The analysis device 3 receives point cloud data from the range measurement sensor 2 via the communication unit 32.

The memory unit 33 is composed of, for example, one or more ROMs, one or more RAMs, and/or an external storage device (USB, SD card, etc.), and stores various information such as analysis programs and other programs executed by the processing unit 31, point cloud data from the range measurement sensor 2, and calculation results from the processing unit 31.

The input unit 34 is composed of, for example, a keyboard, a mouse, and/or push buttons, and receives input from the user.
The display unit 35 is configured with, for example, a liquid crystal panel, and displays various information such as an environmental map (FIG. 14) described later.
The input unit 34 and the display unit 35 may form a touch panel.
The analysis device 3 does not necessarily have to include the input unit 34 and/or the display unit 35 .

[Analysis method]
Next, an analysis method according to one embodiment of the present invention will be described with reference to Fig. 2 and Figs. 4 to 14. Fig. 2 is a schematic flowchart of the analysis method according to one embodiment of the present invention. The following describes a case where the analysis system 1 shown in Fig. 1 is used, but the analysis method of the present invention can also be implemented using an analysis system 1 different from the example shown in Fig. 1.
The analysis method according to this embodiment includes an initial background extraction step S1, a moving object extraction step S2, a movable and stationary object extraction step S3, and an environmental map creation step S4. These steps S1 to S4 are performed by the processing unit 31 of the analysis device 3 by executing an analysis program stored in the storage unit 33.
The range sensor 2 transmits point cloud data to the analysis device 3 in real time via communication (wired communication and/or wireless communication) at each measurement time for each predetermined measurement time interval, and the analysis device 3 performs steps S1 to S3 in real time during that time.
Each step will be explained below.

- Initial background extraction step (S1) -
First, in an initial background extraction step S1, the processing unit 31 performs an initial background extraction process in which one or more stationary objects SO are extracted based on the point cloud data output from the range measurement sensor 2 at each measurement time for each predetermined measurement time interval within a predetermined initial background extraction period, and each cell C constituting each stationary object SO in the cell array CA is registered as an initial background IB.
The length of the predetermined initial background extraction period is preferably set to, for example, 100 to 140 seconds. The predetermined initial background extraction period is preferably set to a relatively early period within the predetermined measurement period of the range sensor 2, and is preferably set so that, for example, the start time of the predetermined measurement period of the range sensor 2 and the start time of the predetermined initial background extraction period coincide with each other.

Figure 6 shows a schematic representation of the analysis data obtained after performing the initial background extraction step S1 (initial background extraction process) on the analysis data of Figure 5. In Figure 6, cell C, registered as the initial background IB, is shown colored gray.

The initial background extraction step S1 includes, for example, an object extraction step S11, a high-position stationary object extraction step S12, and a linear stationary object extraction step S13.

<Object Extraction Step (S11)>
In the object extraction step S11, the processing unit 31 performs an object extraction process to extract one or more objects O based on the point cloud data output from the range measurement sensor 2 at the measurement time within a predetermined initial background extraction period.
The processing unit 31 repeats the object extraction step S11 at each measurement time within a predetermined initial background extraction period.
Specifically, as shown in FIG. 5, in the object extraction step S11, the processing unit 31 performs, for example, the following steps S111 to S112.

First, the processing unit 31 stores the point cloud data output from the range measurement sensor 2 at the most recent measurement time within a predetermined initial background extraction period in the cell array CA (cell array storage step S111). Here, the processing unit 31 stores each point of the point cloud data in a corresponding cell C in the cell array CA based on the x-coordinate and y-coordinate of each point of the point cloud data.
Next, as shown in FIG. 5 , the processing unit 31 extracts a point cloud consisting of multiple adjacent points as a single point cloud cluster and thus as an object O by clustering, and assigns an individual ID to each object O (clustering step S112). When assigning an ID to the object O extracted based on the point cloud data at the current measurement time, if each cell C constituting the object O at least partially overlaps with each cell C constituting any one of the objects O extracted based on the point cloud data at the previous measurement time, the processing unit 31 inherits the ID assigned to the object O extracted based on the point cloud data at the previous measurement time. This enables tracking of the object O between different measurement times. Note that "each cell C constituting the object O" refers to each cell C in which a point cloud constituting the object O exists.
As a specific clustering method, any known method may be used. For reference, in Fig. 5, the ID assigned to each object O extracted in the object extraction step S11 is shown.
In the following, the IDs assigned to the object O may be collectively referred to as "i", "k", etc.

<High-Position Stationary Object Extraction Step (S12)>
In the high-position stationary object extraction step S12, the processing unit 31 performs a high-position stationary object extraction process in which, of the one or more objects O extracted in the object extraction step S11, one or more objects O (hereinafter referred to as "first objects O"), at least a portion of which is located at a height position above a predetermined height, are extracted as stationary objects SO, and each cell C that constitutes each stationary object SO in the cell array CA is registered as an initial background IB.
The processing unit 31 may perform the high-position stationary object extraction step S12 only once using the result (one or more objects O) of the object extraction step S11 performed based on the point cloud data at the first measurement time of the specified initial background extraction period, or may perform the high-position stationary object extraction step S12 using the result of the object extraction step S11 each time the object extraction step S11 is performed.
When the processing unit 31 extracts the first object O as a stationary object SO and registers it as the initial background IB in the high-position stationary object extraction step S12, it cancels the ID assigned to the first object O in the object extraction step S11 (Figure 6).
Here, the predetermined height is preferably set slightly higher than the person P in the predetermined space S, and is preferably set to a height of 1.75 to 1.90 m from the ground, and more preferably set to a height of 1.80 to 1.85 m from the ground. The predetermined height is preferably set to the same height as the installation height of the range sensor 2, which is set slightly higher than the person P in the predetermined space S.
By defining an object O located at a relatively high position as a stationary object SO in the high-position stationary object extraction step S12, the number of points to be processed subsequently can be reduced accordingly, thereby improving the efficiency and real-time nature of the processing by the analysis device 3.

<Straight-line stationary object extraction step (S13)>
In the linear stationary object extraction step S13, the processing unit 31 performs a linear stationary object extraction process in which, for each of one or more objects O (hereinafter referred to as "second objects O") other than the one or more first objects O extracted in the high-position stationary object extraction step S12, among the one or more objects O extracted in the object extraction step S11, a first linearity evaluation value C _cluster (i) representing the degree of linearity of the object is calculated, and one or more second objects O whose first linearity evaluation value C _cluster (i) exceeds a predetermined first linearity evaluation value threshold C _{th_cluster} are extracted as stationary objects SO, and each cell C constituting each stationary object SO in the cell array CA is registered as an initial background IB.
Generally speaking, a stationary object SO extracted by linear stationary object extraction processing can be said to have a surface shape with higher linearity (more linear components) than a person P. Note that, as a method of extracting a stationary object SO, it is possible to extract an object O that has not moved at all during a predetermined initial background extraction period as a stationary object SO, but in that case, there is a risk that a person P who has not moved at all during the predetermined initial background extraction period will also be extracted as a stationary object SO. Therefore, in this example, by focusing on the linearity of the surface shape of the object O instead of focusing on the movement of the object O, such a risk can be avoided and the stationary object SO can be extracted more accurately.

Specifically, in the linear stationary object extraction step S13, the processing unit 31 performs, for example, the following steps S131 to S138 for each second object O.

First, the processing unit 31 stores the point cloud cluster of the second object O obtained as described above from the point cloud data at the measurement time immediately preceding the predetermined initial background extraction period in a virtual voxel array VA (not shown). Here, the processing unit 31 stores each point of the point cloud cluster of the second object O in a corresponding voxel V in the voxel array VA based on the x-coordinate, y-coordinate, and z-coordinate of each point of the point cloud cluster of the second object O (voxel array storage step S131).
The voxel array VA is formed by arranging cubic voxels V (not shown) three-dimensionally along the x-axis, y-axis, and z-axis. The length of one side of the voxels V is the same as the length of one side of the cells C in the cell array CA. The coordinates of each voxel V are expressed as (pq, r) in the xyz Cartesian coordinate system. Here, p, q, and r are preferably integers.

Next, the processing unit 31 calculates the center point P center =(P center:x , P _{center :y} ) on the xy plane of the point cloud stored in any one voxel V (p, q, r) of the voxels V constituting the _{second object} O (center point calculation step S132). The center point P _center specifically means an average point or a center of gravity point. Here, "each voxel V constituting the second object O" refers to each voxel V in which the point cloud constituting the second object O is stored.
Next, the processing unit 31 calculates the angle θn in the xy plane direction between the center point _Pcenter and each point _{Pn =} ( _Pn:x , Pn _:y , Pn _:z ) stored in the voxel using the following formula (1) (angle calculation step S133). This angle _θn is the angle formed by the line segment connecting the pair of points _Pcenter and _Pn , projected onto the xy plane, with respect to the x-axis.

Next, the processing unit 31 calculates the angle standard deviation σ _θ (p, q, r) using the formed angle θ _n and the average value θ _ave of the formed angles θ _n according to the following equation (2) (angle standard deviation calculation step S134).

Next, the processing unit 31 uses the angle standard deviation σ _θ (p, q, r) to calculate the background weight ω _BG (p, q, r) of the voxel V according to the following equation (3) (background weight calculation step S135).
In equation (3), α and β may be set to any value as long as they satisfy the relationship 0<α<β. The inventors of the present invention have found through various analyses that, in general, when a voxel satisfies σ _θ (p,q,r)<0.1, the point cloud within the voxel tends to constitute a stationary object SO, and when a voxel satisfies 0.1≦σ _θ (p,q,r)<0.5, the point cloud within the voxel tends to constitute a moving object MO. In light of this finding, α is preferably set to 0.10, and β is preferably set to 0.50.

Similarly, the processing unit 31 performs steps S132 to S135 for all of the voxels V that make up the second object O to obtain background weights ω _BG (p, q, r) for each of the voxels V.

Next, the processing unit 31 calculates the average value ω _BGave (p, q, r) of the background weights ω _BG (p, q, r) calculated for the second object O (background weight average value calculation step S136).
Similarly, the processing unit 31 performs steps S131 to S136 for each measurement time within the predetermined initial background extraction period to calculate the average value ω _BGave (p,q,r) of each background weight ω _BG (p,q,r) of the second object O.

Next, the processing unit 31 calculates the first linearity evaluation value C cluster (i) of the second object O by adding together the average values ω _BGave (p,q,r) of each background weight ω _BG (p,q,r) of the second object O for each point cloud data at each measurement time within _the specified initial background extraction period (first linearity evaluation value calculation step S137).
The first linearity evaluation value C _cluster (i) represents the degree of linearity of the surface shape of the second object O, and the higher the value, the higher the linearity of the surface shape. The i in C _cluster (i) represents the ID of the second object O.

Next, the processing unit 31 compares the first linearity evaluation value C _cluster (i) with a predetermined first linearity evaluation value threshold C _{th_cluster} , and if the first linearity evaluation value C _cluster (i) exceeds the predetermined first linearity evaluation value threshold C _{th_cluster} , it extracts the second objects O as stationary objects SO and registers each cell C that constitutes each stationary object SO in the cell array CA as an initial background IB (initial background registration step S138).
The predetermined first linearity evaluation value threshold C _{th_cluster} is preferably 960, for example.

When the processing unit 31 extracts the second object O as a stationary object SO and registers it as the initial background IB in the linear stationary object extraction step S13, it cancels the ID assigned to the second object O in the object extraction step S11 (Figure 6).

The initial background extraction step S1 may include only one of the high-position stationary object extraction step S12 and the linear stationary object extraction step S13.
Furthermore, in the initial background extraction step S1, the processing unit 31 may extract the object O as the stationary object SO by a method different from the high-position stationary object extraction step S12 and the linear stationary object extraction step S13 described above.

--Animal object extraction step (S2)--
In the moving object extraction step S2, the processing unit 31 performs a moving object extraction process to extract one or more moving objects MO by subtracting the initial background IB registered in the initial background extraction step S1 from the point cloud data output from the range sensor 2 at the measurement time within a specified observation period after the specified initial background extraction period.
The specified observation period may be set to any period after the specified initial background extraction period within the specified measurement period of the range sensor 2, and is preferably set to, for example, the entire period after the specified initial background extraction period within the specified measurement period of the range sensor 2.
The processing unit 31 repeats the moving object extraction step S2 at each measurement time within a predetermined observation period.
FIG. 7 shows a schematic diagram of point cloud data in the predetermined space of FIG. 4 at a certain measurement time within a predetermined observation period.
Specifically, the processing unit 31 performs the following steps S21 to S23 in the moving object extraction step S2.

7, the processing unit 31 stores the point cloud data output from the range measurement sensor 2 at the most recent measurement time within a predetermined observation period in a cell array CA (cell array storage step S21). Here, the processing unit 31 stores each point of the point cloud data in a corresponding cell C in the cell array CA based on the x-coordinate and y-coordinate of each point of the point cloud data.
Next, the processing unit 31 subtracts the initial background IB registered in the initial background extraction step S1 from the point cloud data, thereby leaving only the point cloud data stored in cells C other than the cell C of the initial background IB as the target for subsequent processing (subtraction step S22). All of this remaining point cloud data becomes point cloud data of the moving object MO.
Next, as shown in FIG. 7 , the processing unit 31 performs clustering on the point cloud data remaining to be processed, extracting a point cloud consisting of multiple adjacent points from the point cloud data as a single point cloud cluster and thus as a moving object MO, and assigns an individual ID to each moving object MO (clustering step S23). When assigning an ID to the moving object MO extracted based on the point cloud data at the current measurement time, if each cell C constituting the moving object MO at least partially overlaps with each cell C constituting one of the moving objects MO extracted based on the point cloud data at the previous measurement time, the processing unit 31 inherits the ID assigned to the moving object MO extracted based on the point cloud data at the previous measurement time. This enables tracking of moving objects MO between different measurement times.
Any known clustering method may be used as a specific clustering method. For reference, in Fig. 7, the ID assigned to each moving object MO extracted in the moving object extraction step S2 is shown.

--Movable and stationary object extraction step (S3)--
In the movable stationary object extraction step S3, the processing unit 31 performs a movable stationary object extraction process in which one or more movable stationary objects MSO are extracted from one or more moving objects MO extracted in the moving object extraction step S2, and each cell C constituting each movable stationary object MSO in the cell array CA is redefined as a post-defined background LB.
The moving object MO that has been redefined as the post-defined background LB is thereafter treated as the post-defined background LB, and is no longer treated as the moving object MO.
The processing unit 31 performs the movable stationary object extraction step S3 every time it performs the moving object extraction step S2 (i.e., at each measurement time within a predetermined observation period).

Fig. 13 shows a schematic diagram of the analysis data obtained after the movable stationary object extraction step S3 (movable stationary object extraction process) is performed on the analysis data of Fig. 7. In Fig. 13, the cell C redefined as the post-defined background LB is shown colored gray.
13, after the movable stationary object extraction step S3, any moving object MO made up of point cloud data stored in cells C of the cell array CA other than cells C of the initial background IB and the post-defined background LB is determined to be a person P. In this way, the analysis system 1 becomes able to distinguish the object O in the predetermined space S into a person P and an object other than a person P.

When the processing unit 31 extracts the moving object MO as a moving stationary object MSO in the moving stationary object extraction step S3 and redefines it as a post-defined background LB, it cancels the ID assigned to the moving object MO in the moving object extraction step S2 (Figure 13).

The movable stationary object extraction step S3 includes, for example, a linked movable stationary object extraction step S31, a linear movable stationary object extraction step S32, and a fine-motion movable stationary object extraction step S34.

<Interlocking Movable Stationary Object Extraction Step (S31)>
Generally, chairs are the most common movable stationary objects (MSO) present in indoor spaces. Chairs tend to move in conjunction with a series of actions, such as when a person sits down, stands up, and then leaves the chair. Therefore, in the linked movable stationary object extraction step S31, such chair and person actions are detected, and chairs are extracted as movable stationary objects (MSO).
In the linked movable stationary object extraction step S31, the processing unit 31 performs a linked movable stationary object extraction process in which, when one or more moving bodies MO (hereinafter referred to as the "first moving body MO") among the one or more moving bodies MO extracted in the moving body extraction step S2 transition from a sitting state to a standing state, and then a new moving body MO is extracted around the first moving body MO by the moving body extraction step S2 (moving body extraction process), and then, within a predetermined new moving body monitoring time from the time when the following formula (5) is first satisfied, the time T _chair (seconds) during which formula (5) is continuously satisfied exceeds a predetermined duration threshold, the processing unit 31 extracts the new moving body MO as a movable stationary object MSO and redefines each cell C that constitutes the movable stationary object MSO in the cell array CA as a post-defined background LB.
In the above formula (5),
h(k,t): Height of the new moving object MO (mm)
γ: A given real number between 0 and 200
h _{sit_ave} (p,q): average sitting height of the first moving body MO (mm)

Specifically, in the linked movable stationary object extraction step S31, the processing unit 31 performs, for example, the following steps S311 to S316.

First, the processing unit 31 determines whether each moving object MO is in a sitting state (sitting state determination step S311). Here, the sitting state is assumed to be a state in which a person is sitting in a chair, while the standing state is assumed to be a state in which a person is standing.
The processing unit 31 performs the sitting position determination step S311 at each measurement time within a predetermined observation period.
Here, a specific method for the sitting position determination step S311 will be described. Each cell C constituting the moving object MO stores the z coordinate value of the highest point among the points contained in that cell C. The height h(i,t) (mm) of the moving object MO (ID is "i") at a certain measurement time t is defined as the z coordinate value of the highest point among the cells C constituting the moving object MO at that measurement time t. The maximum height h _max (i) (mm) of the moving object MO is defined as the maximum value of the height h(i,t) of the moving object MO acquired up to the measurement time t. The sitting position determination threshold h _sit (i) (mm) is defined by the following equation (4):
As shown in Figure 8, the threshold value h _sit (i) for determining the sitting position corresponds to the height of a person in a seated position. In equation (4), 0.55 h _max (i) (mm) corresponds to a person's sitting height. This was determined taking into consideration that, according to a survey by the Ministry of Health, Labor and Welfare, the average sitting height of Japanese people, regardless of gender, is approximately 53-55% of their height. Furthermore, in equation (4), 0.25 h _max (i)-10 (mm) corresponds to the height from the ground to the chair seat. This was determined taking into consideration that the ideal seat height for an ergonomic chair is 25% to 10 mm lower than their height. In situations where a person is walking around, the maximum height h _max (i) of a person approximates their height, so equation (4) uses h _max (i) as the person's height.
In the sitting position determination step S311, if the moving object MO (ID: i) at the most recent measurement time t satisfies h(i, t) < h _sit (i), the processing unit 31 determines that the moving object MO is in a sitting position, otherwise it determines that the moving object MO is in an upright position.

If the processing unit 31 determines in the sitting position determination step S311 that one or more of the moving bodies MO (hereinafter referred to as "first moving bodies MO") are in a sitting position, it stores the height h(i,t) of the first moving body MO at that time in each cell C within a radius of 50 cm from the center P(i) of the first moving body MO (ID: i), and further calculates the average value h _{sit_ave} (p,q) of the heights h(i,t) accumulated in each of these cells C so far (hereinafter referred to as "average sitting height value h _{sit_ave} (p,q)") (average sitting height calculation step S312).
Here, "the center P(i) of the first moving body MO (with ID "i")" means the average point or center of gravity of the group of points that make up the first moving body MO in the xy coordinate system.
The processing unit 31 performs the sitting height average value calculation step S312 for each moving object MO every time it is determined in the sitting position determination step S311 that the moving object MO is in a sitting position.

Furthermore, if the processing unit 31 determines in the sitting position determination step S311 that the first moving body MO is in a sitting position, it continues to monitor the first moving body MO by tracking, and then determines whether the first moving body MO transitions to a standing position (i.e., the first moving body MO is determined to be in a standing position in the subsequent sitting position determination step S311), and then determines whether a new moving body MO (with ID ``k'') is extracted around the first moving body MO in the moving body extraction step S2 (monitoring step S313).
Here, "the vicinity of the first moving body MO" specifically refers to, for example, an area within a radius of 50 cm from the center of each cell C that is within a radius of 50 cm from the center P(i) of the first moving body MO.
In addition, when a new moving object MO (ID: k) is extracted around the first moving object MO in the moving object extraction step S2, it is assumed that a transition has occurred from a state in which the chair and the person sitting on or standing next to it were detected as a single moving object MO (i.e., a state in which only one ID was assigned to the chair and the person; see Figure 6), to a state in which the chair and the person have been detected as separate moving objects MO because the person has left the chair (i.e., a state in which the ID is increased by one and separate IDs are assigned to the chair and the person; see Figure 7).

If the processing unit 31 determines in the monitoring step S313 that the first moving object MO (ID: i) has transitioned from a sitting state to a standing state, and then determines that a new moving object MO (ID: k) has been extracted around the first moving object MO in the moving object extraction step S2, it determines whether the following equation (5) is satisfied (height determination step S314).
In equation (5), h(k,t) is the height (mm) of the new moving object MO, and h _{sit_ave} (p,q) is the average sitting height h _{sit_ave} (p,q) of the first moving object MO (ID:i) calculated in the most recent average sitting height calculation step S312. In equation (5), γ is a predetermined real number equal to or greater than 0 (mm), and is preferably set to, for example, 0 (mm) to 200 (mm), more preferably 50 (mm) to 150 (mm), and is preferably set to, for example, 100 (mm).
It should be noted that the new moving object MO may be a chair that has moved from its initial position, or a person who has just come to sit in the chair. If the new moving object MO is a chair, the height h(k,t) of the new moving object MO is likely to be lower than the average sitting height h _{sit_ave} (p,q) of the first moving object MO. In equation (5), the value obtained by adding γ to the height h(k,t) of the new moving object MO is compared with the average sitting height h _{sit_ave} (p,q) of the first moving object MO, taking into account errors in the range measurement sensor 2 and the like, so as to more reliably determine whether the new moving object MO is a chair.

If the processing unit 31 determines in the height determination step S314 that the formula (5) is satisfied, it measures the time T _chair (seconds) during which the new moving object MO continuously satisfies the formula (5) by treating the new moving object MO as a candidate for a movable stationary object (time measurement step S315).
In the time measurement step S315, if the time T _chair exceeds a predetermined duration threshold within a predetermined new moving object monitoring time from the time when equation (5) is first satisfied, the processing unit 31 extracts the new moving object MO as a movable stationary object MSO and redefines each cell C that constitutes the movable stationary object MSO in the cell array CA as a post-defined background LB (redefinition step S316).
The predetermined new moving object monitoring time is preferably, for example, 100 to 140 seconds, and more preferably, 120 seconds, and the predetermined duration threshold is preferably, for example, 5 to 20 seconds, and more preferably, 12 seconds.

When the processing unit 31 extracts the new moving object MO as a movable stationary object MSO and redefines it as a post-defined background LB in the redefinition step S316, it cancels the ID assigned to the new moving object MO in the moving object extraction step S2 (Figure 13).

<Straight-line movable stationary object extraction step (S32)>
Generally, among the movable stationary objects MSO present in an indoor space, the second most common after chairs are whiteboards, desks, etc. Most of these have flat surfaces and are represented as a collection of linear points in the point cloud data obtained from the range sensor 2. Therefore, in the linear movable stationary object extraction step S32, the linear shape of the surface of the moving object MO is extracted, a second linearity evaluation value is calculated based on the extracted linear shape, and the whiteboard, desk, etc. are extracted as the movable stationary object MSO based on the second linearity evaluation value.
In the linear movable stationary object extraction step S32, when an occlusion area SR appears in the initial background IB at the same location over a predetermined occlusion area monitoring time, the processing unit 31 identifies one or more moving objects MO extracted in the moving object extraction step S2 that is located between the range sensor and the occlusion area SR (hereinafter referred to as the "second moving object MO"), calculates a second linearity evaluation value C _BG (t,i) for the second moving object MO that represents the degree of linearity of the object, and if the second linearity evaluation value C _BG (t,i) exceeds a predetermined second linearity evaluation value threshold C _{th_BG} , extracts the second moving object MO as a movable stationary object MSO, and redefines each cell C that constitutes the movable stationary object MSO in the cell array CA as a post-defined background LB, thereby performing a linear movable stationary object extraction process.

Specifically, in the linear movable stationary object extraction step S32, the processing unit 31 performs, for example, the following steps S321 to S331.

First, the processing unit 31 determines whether or not a shading region SR has appeared in the initial background IB at the same location for a predetermined shading region monitoring time based on the point cloud data stored in the cell array CA (shading region monitoring step S321).
If the movable stationary object MSO is moved by a person during the predetermined observation period, an occluded region SR (shadow) is generated in the initial background IB in the cell array CA by the movable stationary object MSO. The occluded region SR is an area that is not detected by the range sensor 2 due to the movable stationary object MSO being between the range sensor 2 and the occluded region SR, and is an area in the cell array CA where no point cloud exists. For example, in the example of FIG. 7 , a whiteboard serving as the movable stationary object MSO (ID: 8) is moved into the predetermined space S during the predetermined observation period, and as a result, an occluded region SR is generated in the initial background IB by the whiteboard.
Specifically, in the occlusion area monitoring step S321, if at least one cell C among the cells C that constitute the initial background IB does not contain a point cloud over a specified occlusion area monitoring time, the processing unit 31 determines that an occlusion area SR consisting of the at least one cell C has appeared in the initial background IB at the same location over the specified occlusion area monitoring time.

If the processing unit 31 determines in the shading area monitoring step S321 that a shading area SR has appeared in the initial background IB at the same location for a predetermined shading area monitoring time, it identifies, among each moving body MO, the moving body MO that is located between the range sensor 2 and the shading area SR (hereinafter referred to as the "second moving body MO") (second moving body identification step S322).
It should be noted that the second moving object MO may be the movable stationary object MSO that caused the occlusion region SR. Therefore, the second moving object MO is set as a candidate for the movable stationary object MSO, and the following processing is performed on the second moving object MO.
It should be noted that, if the processing unit 31 has identified a plurality of second moving objects MO for each of a plurality of shield regions SR in steps S321 to S322, the processing unit 31 performs the following processing for each of the second moving objects MO.

Next, the processing unit 31 calculates the center point P _cell of each cell C(p,q) constituting the second moving object MO (cell center point calculation step S323), as shown in Fig. 9. Here, the center point P _cell of a cell C(p,q) means the average point or center of gravity of the point cloud stored in the cell C. By performing the cell center point calculation step S323, it is possible to reduce the number of point clouds to be processed thereafter, thereby improving the efficiency and real-time nature of the processing.

Next, the processing unit 31 sorts the multiple center points P _cells that make up the second moving body MO in ascending order of the x-coordinates of the center points P _cells , and calculates the Euclidean distance d _n (Figure 9) between each pair of adjacent center points P _cells for each pair of adjacent center points P _cells (Euclidean distance calculation step S324).
In the Euclidean distance calculation step S324, the processing unit 31 preferably sorts the multiple center points P _cells constituting the second moving object MO in ascending order of the x-coordinates of the P _cells and calculates the Euclidean distance _dn (the Euclidean distance dn between the center point P _cell and the other center point P _{cell closest to the center point P cell )} starting from the center point P _cell at the edge (the P _cell with the smallest x-coordinate). In this case, only one Euclidean distance _dn is obtained for each center point P _cell (for each center point P _cell ). If multiple center points P _cells exist at the same _x -coordinate, the multiple center points P _cells are sorted in ascending order of the y-coordinates of the center point P _cells , and the Euclidean distance _dn between pairs of center points P _cells adjacent to each other in the y-axis direction is calculated for each pair of adjacent center points P _cells . Note that n (n = 2, 3, ... N) indicates the number of cells C constituting the second moving object MO.

Next, the processing unit 31 calculates the average value _{dn_ave} of each Euclidean distance _dn of the second moving object MO, and divides the multiple center points P _cells constituting the second moving object MO into two divided clusters using the average value _{dn_ave} , as shown in Fig. 10 (cluster division step S325). At this time, one divided cluster (hereinafter referred to as the "first divided cluster") is made up of center points P _cells corresponding to Euclidean distances _dn less than the average value _{dn_ave} among the multiple center points P _cells constituting the second moving object MO, and the other divided cluster (hereinafter referred to as the "second divided cluster") is made up of center points P _cells corresponding to Euclidean distances _dn equal to or greater than the average value _{dn_ave} among the multiple center points P _cells constituting the second moving object MO.
The inventors of the present invention have found through various analyses that the distribution of Euclidean distances _dn for movable and stationary objects MSO having highly linear surface shapes generally has two maximum values (peaks), as shown in Fig. 10. The cluster division step S325 is based on this finding. Here, the "distribution of Euclidean distances _dn " refers to a graph, as shown in Fig. 10, in which the horizontal axis represents the Euclidean distance _dn and the vertical axis represents the ratio of the number of _dn having each value to the total number of _dn .

Next, for each of the first and second divided clusters, the processing unit 31 calculates the angle θ m ( FIG. 11 ) between a pair of adjacent center points P _cell =(P _{cell ,x:k} , P _cell,y:k ), P _cell =(P _cell,x:k+1 , P _cell,y:k+1 ) among the multiple center points P cell that make up each divided cluster, for each pair of adjacent center points P _cell , using the following equation (6) (angle calculation step S326). This angle θ _m is the angle formed by the line segment connecting the pair of adjacent center points P _{cell with} respect to the x-axis.
Here, m (m=1, 2...M) indicates the number of cells that make up the divided cluster, and M indicates the number of cells that make up the divided cluster (i.e., the number of center point P _cells that make up the divided cluster).

Next, the processing unit 31 calculates the straightness μ (degrees) for each of the first divided cluster and the second divided cluster using the angle θ _m calculated in the angle calculation step S326 according to the following equation (7) (straightness calculation step S327).
The closer the linearity μ is to 0, the higher the linearity of the surface shape.

Next, the processing unit 31 calculates a shape score ε for each of the first divided cluster and the second divided cluster using the straightness μ calculated in the straightness calculation step S327 according to the following equation (8) (shape score calculation step S328).
In equation (8), α = -0.0006 and β = 5.0. As can be seen from equation (8), the shape score ε is expressed as a quadratic function. Based on the finding that the distribution of straightness μ can be approximated by an inverse proportional graph, the equation for the shape score ε was set in this way.

Next, the processing unit 31 determines the larger of the shape scores ε calculated for each of the first and second divided clusters in the shape score calculation step S328 as the large shape score ε _max , and the smaller of the shape scores ε as the small shape score ε _min (large and small shape score determination step S329).

Next, the processing unit 31 calculates the second linearity evaluation value C _BG (t,i) at the most recent measurement time t using the following equation (9) (second linearity evaluation value calculation step S330).
The inventors of the present invention have found through various analyses that, in general, movable and stationary objects MSO having highly linear surface shapes tend to have 0 < ε _max < 2.0 and 0 < ε _min < 1.5, while humans tend to have 3.0 < ε _max < 5.0 and 2.0 < ε _min < 4.0. Formula (9) for the second linearity evaluation value C _BG (t,i) was determined based on this finding.
The higher the second linearity evaluation value C _BG (t,i), the higher the linearity of the surface shape of the second moving object MO, and therefore the higher the possibility that the second moving object MO is a movable stationary object MSO.
The processing unit 31 repeats the above-described steps S323 to S330 at every predetermined measurement time interval (that is, at every measurement time t) to calculate the second linearity evaluation value C _BG (t,i).
In parallel with this, the processing unit 31 monitors the speed of the second moving object MO, and if the speed of the second moving object MO becomes 0.1 m/s or more for more than 1 second, it is highly likely that the second moving object MO is a person, and therefore, in the second linearity evaluation value calculation step S330, it is preferable to subtract 1.0 from the second linearity evaluation value C _BG (t,i) calculated by equation (9).

If the second linearity evaluation value C _BG (t, i) exceeds a predetermined second linearity evaluation value threshold C _{th_BG} within a predetermined second linearity evaluation time from the time the second moving object MO is identified in the second moving object identification step S322, the processing unit 31 extracts the second moving object MO as a movable stationary object MSO and redefines each cell C that constitutes the movable stationary object MSO in the cell array CA as a post-defined background LB (redefinition step S331).
The predetermined second linearity evaluation time is preferably, for example, 100 to 140 seconds, and is preferably, for example, 120 seconds. The predetermined second linearity evaluation value threshold C _{th_BG} is preferably, for example, 960.

When the processing unit 31 extracts the second moving object MO as a movable stationary object MSO and redefines it as a post-defined background LB in the redefinition step S331, it cancels the ID assigned to the second moving object MO in the moving object extraction step S2 (Figure 13).

<Slightly Moving Stationary Object Extraction Step (S34)>
Generally, indoor movable stationary objects MSO include not only the movable stationary objects MSO extracted in the above-mentioned linked movable stationary object extraction step S31 and linear movable stationary object extraction step S32, but also other movable stationary objects MSO that vary in shape depending on the environment and are moved at different times and under different conditions, such as houseplants, trash cans, and laptops. The greatest difference between these movable stationary objects MSO and people is whether or not they actively move. While people can actively move, these movable stationary objects MSO do not move actively, and therefore their movement is slight. Therefore, in the slight-movement movable stationary object extraction step S34, moving objects MO that move only slightly are extracted as movable stationary objects MSO.

Step S34 for extracting a slightly moving movable stationary object includes, for example, step S341 for extracting a maximum point slightly changing movable stationary object and step S342 for extracting a slightly changing voxel number movable stationary object.

<Highest point slight fluctuation type movable stationary object extraction step (S341)>
In the maximum point slight fluctuation type movable stationary object extraction step S341, focusing on the amount of movement of the maximum point of the moving object MO (and therefore the position distribution of the maximum point), a moving object MO with slight movement is extracted as a movable stationary object MSO.
In the highest point slight fluctuation type movable stationary object extraction step S341, the processing unit 31 performs a highest point slight fluctuation type movable stationary object extraction process in which, if the highest point of one or more moving bodies MO (hereinafter referred to as the "third moving body MO") among the one or more moving bodies MO extracted in the moving body extraction step S2 is located at approximately the same position over a predetermined highest point monitoring time, the processing unit 31 extracts the third moving body MO as a movable stationary object MSO and redefines each cell C that constitutes the movable stationary object MSO in the cell array CA as a post-defined background LB.

As described above in the description of the sitting position determination step S311, each cell C constituting the moving object MO stores the z coordinate value of the highest point (highest point) among the group of points contained in that cell C. The height h(i, t) (mm) of the moving object MO (with ID "i") at the most recent measurement time t is defined as the z coordinate value of the highest point (highest point) among the cells C constituting the moving object MO at that measurement time t.
In the highest point slight fluctuation type movable stationary object extraction step S341, the processing unit 31 performs, for example, the following steps S3411 to S3412.

First, for each moving body MO, the processing unit 31 stores the position (p, q) of one or more cells C having the height h(i, t) (highest point) of the moving body MO for a predetermined highest point cell storage time (highest point cell storage step S3411).
Here, the predetermined highest point cell storage time is preferably, for example, 100 to 140 seconds, and is preferably, for example, 120 seconds.

Next, if one or more of the one or more moving bodies MO (hereinafter referred to as "third moving bodies MO") have one or more cells C that have been consistently saved for the specified highest point cell saving time in the highest point cell saving step S3411, the processing unit 31 determines that the highest points of these third moving bodies MO have been located in approximately the same position for the specified highest point monitoring time, extracts each third moving body MO as a movable stationary object MSO, and redefines each cell C that constitutes the movable stationary object MSO in the cell array CA as a post-defined background LB (redefinition step S3412).

When the processing unit 31 extracts the third moving object MO as a movable stationary object MSO and redefines it as a post-defined background LB in redefinition step S3412, it cancels the ID assigned to the third moving object MO in moving object extraction step S2 (Figure 13).

<Voxel number slight variation type movable stationary object extraction step (S342)>
The position distribution of the highest points of the moving objects MO focused on in the highest point slight variation type movable stationary object extraction step S341 may vary even for movable stationary objects MSO that are not actively moving, when the highest point position exists on the boundary between cells C or due to the influence of errors in the range measurement sensor 2. To enable the extraction of movable stationary objects MSO even in such cases, the voxel number slight variation type movable stationary object extraction step S342 focuses on the variation in the number of voxels constituting the moving object of the moving object MO and extracts moving objects MO with slight movement as movable stationary objects MSO.
In the voxel number slight variation type movable stationary object extraction step S342, if one or more of the one or more moving bodies MO (hereinafter referred to as the "fourth moving body MO") extracted in the moving body extraction step S2 have a voxel number variation evaluation value representing the degree of variation in the number of voxels V(i,t) constituting the moving body of the fourth moving body MO within a predetermined voxel number monitoring time t _th that is less than a predetermined voxel number variation evaluation value threshold, the processing unit 31 performs a voxel number slight variation type movable stationary object extraction process in which the fourth moving body MO is extracted as a movable stationary object MSO and each cell C constituting the movable stationary object MSO in the cell array CA is redefined as a post-defined background LB.

As described above in the description of the sitting position determination step S311, each cell C constituting the moving object MO stores the z coordinate value of the highest point (highest point) among the points included in that cell C.
In the voxel number slight variation type movable stationary object extraction step S342, the processing unit 31 performs, for example, the following steps S3421 to S3424.

First, the processing unit 31 calculates the number of voxels V(i,t) constituting the moving body MO _at the most recent measurement time t using the following equations (10) and (11) at predetermined measurement time intervals (i.e., at each measurement time t) over a predetermined voxel number monitoring time tth (moving body voxel number calculation step S3421).
The number of voxels constituting the moving body V(i, t) represents the number of voxels V estimated to constitute the moving body MO in the voxel array VA.
Here, the predetermined number of voxels monitoring time t _th is preferably, for example, 100 to 140 seconds, and is preferably, for example, 120 seconds.
In equation (11), V _cell (n,t) is the number of cell-corresponding voxels, and represents the number of voxels V when a voxel V with a side length of l _th is arranged along the z-axis between the highest point of the point cloud in any one of the cells C(p,q) constituting the moving body MO in the cell array CA and the ground, regardless of whether there is a point cloud or not, and is a value assigned to each cell C constituting the moving body MO. In equation (11), h _max (p,q) is the maximum height (height of the highest point) of the point cloud contained in the cell C(p,q), l _th is the length of one side of the voxel V, and n is the number of cells C constituting the moving body MO (ID:i) in the cell array CA. The length l _th of one side of the voxel V is the same as the length of one side of the cell C in the cell array CA.
The number of voxels V(i,t) constituting the moving body defined by equations (10) and (11) is the number of voxels V when voxels V with a side length of l _th are arranged along the z-axis between the highest point stored in each cell C constituting the moving body MO in the cell array CA and the ground, regardless of the presence or absence of a point cloud. For example, in the example very roughly shown in Figure 12, the number of voxels V(i,t) constituting the moving body is the number of voxels V with diagonal hatching.
By defining the number of voxels V(i,t) constituting the moving body in this manner, even if a portion of the point cloud (e.g., the lower half) of the moving body MO is missing in the point cloud data from the range sensor 2, it becomes possible to represent the size of the moving body MO in voxels V in a pseudo manner.

Next, the processing unit 31 calculates the amount of change in the number of voxels V _dif (i, t) by the following equation (12) (voxel number change calculation step S3422).
The amount of change in the number of voxels V _dif (i, t) represents how much the number of voxels constituting the moving body V(i, t) has changed during the predetermined voxel number monitoring time t _th .
In equation (12), V _mode (i) is the mode value of the number of voxels V(i,t) constituting the moving body calculated in the moving body voxel number calculation step S3421 during the predetermined voxel number monitoring time t _th .
When calculating the change in the number of voxels V _dif (i, t), instead of comparing it with the previous number of voxels V(i, t) constituting the moving body, the difference with the mode V _mode (i) is taken as in equation (12), thereby suppressing the influence of errors in the range measurement sensor 2 and making it possible to more clearly grasp the degree of fluctuation in the number of voxels V(i, t) constituting the moving body due to the active movement of a person.

Next, the processing unit 31 calculates the voxel number fluctuation evaluation value σ _dif (i), which is the standard deviation of the voxel number change amount V _dif (i, t) for the specified voxel number monitoring time t _th , using the following equation (13) (voxel number fluctuation evaluation value calculation step S3423).
In equation (13), V _{dif — ave} (i) is the average value of the voxel number change amount V _dif (i, t).
The voxel count fluctuation evaluation value σ _dif (i) represents the degree of fluctuation in the number of voxels V(i,t) constituting the moving body of the fourth moving body MO within a specified voxel count monitoring time t _th , and the higher the value, the higher the degree of fluctuation in the number of voxels V(i,t) constituting the moving body, and ultimately the more likely it is that the fourth moving body MO is a person P.

Next, the processing unit 31 determines whether the fourth moving object MO is a movable stationary object MSO based on the voxel number fluctuation evaluation value σ _dif (i) calculated in the voxel number fluctuation evaluation value calculation step S3423, and if it determines that the fourth moving object MO is a movable stationary object MSO, it extracts the fourth moving object MO as a movable stationary object MSO and redefines each cell C that constitutes the movable stationary object MSO in the cell array CA as a post-defined background LB (redefinition step S3424).
In the redefinition step S3424, the processing unit 31 determines that the fourth moving object MO is a movable stationary object MSO if σ _dif (i)<5.0, where 5.0 in this example corresponds to the predetermined voxel number fluctuation evaluation value threshold.
In the redefinition step S3424, if 5.0≦σ _dif (i)<6.0, the processing unit 31 performs the above steps S3421 to S3423 again for half the specified voxel number monitoring time t _th (t _th /2), and determines that the fourth moving object MO is a movable stationary object MSO only if the voxel number fluctuation evaluation value σ _dif (i) obtained again thereby satisfies σ _dif (i)<5.0.
In the redefinition step S3424, the processing unit 31 determines that the fourth moving object MO is not a movable stationary object MSO (that is, a person P) if 6.0≦σ _dif (i).
The inventors of the present invention have found through various analyses that if σ _dif (i)<5.0, the moving object MO is a movable stationary object MSO, if 6.0≦σ _dif (i), the moving object MO is a person P, and if 5.0≦σ _dif (i)<6.0, the moving object MO is with a high probability a person P, but may also be a movable stationary object MSO. The redefinition step S3424 described above was set based on such findings.

When the processing unit 31 extracts the fourth moving object MO as a movable stationary object MSO and redefines it as a post-defined background LB in redefinition step S3424, it cancels the ID assigned to the fourth moving object MO in moving object extraction step S2 (Figure 13).

As can be seen from Figure 13, after the movable stationary object extraction step S3, all moving objects MO made up of point cloud data stored in cells C of the cell array CA other than cells C of the initial background IB and the post-defined background LB are determined to be people P. After the movable stationary object extraction step S3, the ID is assigned only to people P. In this way, the analysis system 1 becomes able to distinguish objects O in the specified space S into people P and non-human objects.

From the viewpoint of improving processing efficiency and real-time performance, it is preferable to first perform the linked movable stationary object extraction step S31, then the linear movable stationary object extraction step S32, and then the fine-motion movable stationary object extraction step S34 in the movable stationary object extraction step S3.
However, the linked movable stationary object extraction step S31, the linear movable stationary object extraction step S32, and the fine-motion movable stationary object extraction step S34 may be performed in any order, or two or three of them may be performed simultaneously in parallel.
Furthermore, from the viewpoint of improving processing efficiency and real-time performance, it is preferable that the fine-movement type movable stationary object extraction step S34 first performs the highest point fine-movement type movable stationary object extraction step S341, and then performs the voxel number fine-movement type movable stationary object extraction step S342.
However, the highest point slight variation type movable stationary object extraction step S341 and the voxel number slight variation type movable stationary object extraction step S342 may be performed in the reverse order, or may be performed simultaneously in parallel.

In addition, the movable stationary object extraction step S3 may include any one or two of the above-mentioned linked movable stationary object extraction step S31, linear movable stationary object extraction step S32, and fine-motion movable stationary object extraction step S34.
Furthermore, the slight movement type movable stationary object extraction step S34 may include only one of the above-mentioned highest point slight fluctuation type movable stationary object extraction step S341 and the voxel number slight fluctuation type movable stationary object extraction step S342.
Furthermore, in the movable stationary object extraction step S3, the processing unit 31 may extract the moving object MO as the movable stationary object MSO by a method different from the above-described method.

--Environmental map creation step (S4)--
In the environmental map creation step S4, the processing unit 31 creates an environmental map K based on the analysis results of the point cloud data for each measurement time within the predetermined observation period obtained in the movable/stationary object extraction step S3.
The environmental map creation step S4 may be performed after the movable stationary object extraction step S3, or may be performed in parallel with the movable stationary object extraction step S3.
The environmental map K preferably represents how a predetermined space S is used by people. The above-described steps S1 to S3 make it easy to grasp the number and positions of people P at each measurement time within a predetermined observation period, as well as the positions and shapes of objects other than people P registered as the initial background IB and the post-defined background LB, making it possible to more accurately represent how the predetermined space S is used by people. The processing unit 31 recognizes objects O to which IDs have been assigned as people P.
FIG. 14 shows an example of the environmental map K.
14, it is preferable that the environmental map K is represented using a cell array CA. In addition, it is preferable that in the cell array CA, the cells C of the initial background IB and the later-defined background LB are displayed using a different notation (such as a color or hatching) from the other cells C.
For example, as in the example of Fig. 14 , the environmental map K may display, in each cell C other than the initial background IB and the later-defined background LB, the extent to which the space of the cell C was utilized by the person P within a predetermined observation period (degree of activity) using any notation method (e.g., color intensity or type of hatching, etc.). In the example of Fig. 14 , the initial background IB and the later-defined background LB are displayed in black, and in each cell C other than the initial background IB and the later-defined background LB, the higher the degree of activity of the cell C, the darker the gray color displayed.
Additionally, for example, the environmental map K may display, in each cell C other than the initial background IB and the later-defined background LB, the direction of movement of person P, the speed of person P, or the acceleration of person P in the space of that cell C within a specified observation period using any notation method (for example, color intensity or type of hatching, etc.).
As a specific method for the environmental map creation step S4, for example, the method described in Japanese Patent Application No. 2020-038043 may be used.

As explained above, according to the above-mentioned embodiment, the processing unit 31 performs an initial background extraction step S1 (initial background extraction processing) for extracting stationary objects SO, a moving object extraction step S2 (moving object extraction processing) for extracting moving objects MO, and a movable stationary object extraction step S3 (movable stationary object extraction processing) for extracting movable stationary objects MSO, making it possible to distinguish objects O in the space S into people P and non-human objects (stationary objects SO and movable stationary objects MSO).

Furthermore, according to the above-described embodiment, the processing is efficient, so that the processing of the initial background extraction step S1 and the processing from the moving object extraction step S2 to the movable stationary object extraction step S3 can be performed within a predetermined measurement time interval (e.g., 0.1 seconds), enabling real-time processing. Therefore, this analysis system and analysis method can be suitably used for mobile robot navigation, etc.
Furthermore, according to the above-described embodiment, in the initial background extraction step S1 and the movable/stationary object extraction step S3, objects are extracted by focusing on characteristics common to multiple types of objects. Therefore, there is no need to perform machine learning in advance for each object other than a person, which has individually different shapes, movements, etc., and this analysis system and analysis method can be used in a variety of environments.

The analysis system and analysis method of the present invention are suitable for use in analyzing indoor spaces, such as office spaces and school spaces.

1 Analysis system 2 Range measurement sensor 3 Analysis device 31 Processing unit 32 Communication unit 33 Memory unit 34 Input unit 35 Display unit CA Cell array C Cell VA Voxel array V Voxels O, Oa to Og Object SO Stationary object MO Moving object MSO Movable stationary object P Person S Predetermined space IB Initial background LB Post-defined background SR Obscured area K Environmental map

Claims (9)

a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis system comprising:
The processing unit
an initial background extraction process for extracting one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registering, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction process for extracting one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction process for extracting one or more movable stationary objects from the one or more moving objects and redefining each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
and
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
The processing unit, in the movable stationary object extraction process,
a linked movable stationary object extraction process in which a first moving object among the one or more moving objects transitions from a sitting state to a standing state, and then a new moving object is extracted around the first moving object by the moving object extraction process, and then, if the time during which the following formula (5) is continuously satisfied exceeds a predetermined duration threshold within a predetermined new moving object monitoring time from the time when the formula (5) is first satisfied, the new moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background;
an analysis system configured to :
In the above formula (5),
h(k,t): Height of the new moving object (mm)
γ: A given real number between 0 and 200
h _{sit_ave} (p, q): average sitting height of the first moving object (mm) a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis system comprising:
The processing unit
an initial background extraction process for extracting one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registering, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction process for extracting one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction process for extracting one or more movable stationary objects from the one or more moving objects and redefining each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
and
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
The processing unit, in the movable stationary object extraction process,
a linear-type movable stationary object extraction process in which, when an obstructed area appears in the initial background at the same location over a predetermined obstructed area monitoring time, a second moving object located between the range measurement sensor and the obstructed area is identified from among the one or more moving objects, a second linearity evaluation value representing the degree of linearity of the object is calculated for the second moving object, and when the second linearity evaluation value exceeds a predetermined second linearity evaluation value threshold, the second moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a later-defined background;
an analysis system configured to: a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis system comprising:
The processing unit
an initial background extraction process for extracting one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registering, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction process for extracting one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction process for extracting one or more movable stationary objects from the one or more moving objects and redefining each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
and
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
The processing unit, in the movable stationary object extraction process,
When the highest point of a third moving object among the one or more moving objects is located at approximately the same position over a predetermined highest point monitoring time, the third moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background.
an analysis system configured to: a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis system comprising:
The processing unit
an initial background extraction process for extracting one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registering, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction process for extracting one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction process for extracting one or more movable stationary objects from the one or more moving objects and redefining each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
and
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
The processing unit, in the movable stationary object extraction process,
a voxel number slight variation type movable stationary object extraction process, in which, when a voxel number variation evaluation value representing the degree of variation in the number of voxels constituting the fourth moving object of the one or more moving objects is less than a predetermined voxel number variation evaluation value threshold, the fourth moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background;
an analysis system configured to: The processing unit, in the initial background extraction process,
an object extraction process for extracting one or more objects based on the point cloud data output from the range measurement sensor at the measurement time within the predetermined initial background extraction period;
a high-position stationary object extraction process for extracting, as stationary objects, one or more first objects at least a portion of which is located at a height position equal to or higher than a predetermined height from among the one or more objects extracted by the object extraction process, and registering, in the cell array, each cell constituting each of the stationary objects as the initial background;
a linear stationary object extraction process that calculates a first linearity evaluation value representing the degree of linearity of one or more second objects other than the one or more first objects extracted by the high-position stationary object extraction process among the one or more objects extracted by the object extraction process, extracts one or more second objects whose first linearity evaluation value exceeds a predetermined first linearity evaluation value threshold as a stationary object, and registers each cell constituting each of the stationary objects in the cell array as the initial background;
The analysis system according to any one of claims 1 to 4 , configured to perform the following: a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis method using an analysis system comprising:
an initial background extraction step in which the processing unit extracts one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registers, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction step in which the processing unit extracts one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction step in which the processing unit extracts one or more movable stationary objects from the one or more moving objects, and redefines each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
Including,
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
In the movable stationary object extraction step, the processing unit
a linked movable stationary object extraction step in which a first moving object among the one or more moving objects transitions from a sitting state to a standing state, and then a new moving object is extracted around the first moving object by the moving object extraction step, and then, when a time during which the following formula (5) is continuously satisfied exceeds a predetermined duration threshold within a predetermined new moving object monitoring time from the time when the formula (5) is first satisfied, the new moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background;
This is an analysis method.
In the above formula (5),
h(k,t): Height of the new moving object (mm)
γ: A given real number between 0 and 200
h _{sit_ave} (p, q): average sitting height of the first moving object (mm) a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis method using an analysis system comprising:
an initial background extraction step in which the processing unit extracts one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registers, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction step in which the processing unit extracts one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction step in which the processing unit extracts one or more movable stationary objects from the one or more moving objects, and redefines each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
Including,
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
In the movable stationary object extraction step, the processing unit
a linear movable stationary object extraction step in which, when an obstructed area appears in the initial background at the same location over a predetermined obstructed area monitoring time, a second moving object located between the range measurement sensor and the obstructed area is identified from among the one or more moving objects, a second linearity evaluation value representing the degree of linearity of the object is calculated for the second moving object, and when the second linearity evaluation value exceeds a predetermined second linearity evaluation value threshold, the second moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a later-defined background.
This is an analysis method. a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis method using an analysis system comprising:
an initial background extraction step in which the processing unit extracts one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registers, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction step in which the processing unit extracts one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction step in which the processing unit extracts one or more movable stationary objects from the one or more moving objects, and redefines each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
Including,
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
In the movable stationary object extraction step, the processing unit
a highest point slight fluctuation type movable stationary object extraction step in which, when the highest point of a third moving object among the one or more moving objects is located at approximately the same position over a predetermined highest point monitoring time, the third moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background.
This is an analysis method. a range sensor that measures a distance to an object in a predetermined space at each measurement time for each predetermined measurement time interval and outputs three-dimensional point cloud data;
a processing unit;
An analysis method using an analysis system comprising:
an initial background extraction step in which the processing unit extracts one or more stationary objects based on the point cloud data output from the range measurement sensor at each of the measurement times within a predetermined initial background extraction period, and registers, as an initial background, each cell constituting each of the stationary objects in a cell array defined by overlooking the predetermined space and subdividing it into a grid;
a moving object extraction step in which the processing unit extracts one or more moving objects by subtracting the initial background from the point cloud data output from the range measurement sensor at the measurement time within a predetermined observation period after the predetermined initial background extraction period;
a movable stationary object extraction step in which the processing unit extracts one or more movable stationary objects from the one or more moving objects, and redefines each cell constituting each of the movable stationary objects in the cell array as a post-defined background;
Including,
the processing unit recognizes, as a person, any of the moving objects formed by point cloud data stored in cells other than the cells of the initial background and the post-defined background in the cell array;
The movable stationary object refers to an object that moves in response to interaction with a person,
In the movable stationary object extraction step, the processing unit
a voxel number slight variation type movable stationary object extraction step, in which, when a voxel number variation evaluation value representing the degree of variation in the number of voxels constituting the fourth moving object of the one or more moving objects is less than a predetermined voxel number variation evaluation value threshold, the fourth moving object is extracted as a movable stationary object, and each cell constituting the movable stationary object in the cell array is redefined as a post-defined background;
This is an analysis method.