JP7598779B2 - Congestion rate measurement system and program - Google Patents

Description

The present invention relates to a congestion rate measurement system and program that measures the congestion rate, which indicates the degree of congestion on a train.

The congestion rate of a train (also called the occupancy rate) is calculated by dividing the number of passengers by the transportation capacity, and is 100% when the train is at capacity. The congestion rate is roughly 100% when "you can sit in a seat, hold on to a strap, or hold on to a pillar near the door", 150% when "you can read a newspaper easily by spreading it out", 180% when "you can read a newspaper if you make an effort such as folding it", 200% when "your bodies are touching each other and there is a considerable sense of pressure, but you can still read a weekly magazine", and 250% when "every time the train shakes your body becomes tilted and you can't move, and you can't even move your hands". The congestion rate of trains is required to be measured several times a year as a guideline of the Ministry of Land, Infrastructure, Transport and Tourism. Previously, people would go to the stations to be measured, visually check the condition of the train interior, and judge the congestion rate against the above guidelines. However, since the judgment is made by a human being through visual inspection, there are problems such as individual differences that occur, it is difficult to measure every car in the same train, and it is not possible to measure every car and train throughout the year, so only sample measurements can be made.

Methods that were considered included installing cameras near the doors of trains to count the number of passengers getting on and off, and installing cameras near the stairs at station platforms to count the number of people using the stairs, but the former would require one camera for each door, and the latter could capture the number of people entering and exiting the platform but would not be able to measure the occupancy rate of each train, and in the case of shared platforms it would not be possible to tell which train passengers were on, making it unrealistic. A method was also proposed in which cameras would be installed inside trains to count the number of passengers, but this would require a camera in each car and there would be many passengers in the camera's blind spot, making accurate counting impossible. Furthermore, it has been pointed out that these methods using cameras could raise issues in terms of protecting privacy and personal information, including portrait rights.

Furthermore, the weight of the vehicle measured by a sensor, known as adaptive load, is sometimes used to calculate the congestion rate (Patent Documents 1 and 2). Patent Document 1 discloses a method that uses adaptive load data, which calculates the passenger load by estimating the number of passengers and calculating the occupancy rate based on weight data measured by the load detection mechanism of a adaptive load device used to correct the driving force or braking force when accelerating or decelerating a train. Patent Document 2 discloses that in a railway information distribution system that provides an information distribution service according to the user's wishes and uses the user's wishes in operation management, the number of passengers measured by a weight sensor is sent to the operation management system via a wireless device, a wireless base station, and a network, and the degree of congestion on the train is grasped and referred to when rescheduling operations or creating operation information.

A system for measuring passenger occupancy rates has also been proposed (Patent Documents 3 and 4). Patent Document 3 discloses an occupancy rate measurement system that has an imaging device that captures images of trains entering or exiting a station platform, and an image processing device that combines multiple images captured by the imaging device in chronological order to generate an image for each vehicle, and in which a user looks at the image for each vehicle, determines the occupancy rate of the corresponding vehicle, and inputs the occupancy rate. Furthermore, Patent Document 3 discloses that a function may be provided to extract the number of edge pixels (edge amount), the area of the human area in the image, or the number of corners as feature amounts from the image of the vehicle (particularly the image portion within the window frame through which the interior of the vehicle can be seen), and to learn the relationship between the image of the vehicle and the occupancy rate and automatically calculate the occupancy rate.

Patent document 4 discloses a system that uses a laser range finder to scan the side of a train, detects the three-dimensional shape of the interior of the vehicle from laser light that passes through the windows and enters the vehicle, and uses pattern recognition of the three-dimensional shape to distinguish passengers and count the number of passengers present in the vehicle to detect the degree of congestion. Patent document 4 also discloses that a camera or infrared camera is used as a measuring unit, and the degree of congestion is determined by pattern recognition within an image or the area of an area within a frame that indicates the presence of passengers.

Japanese Unexamined Patent Publication No. 60-8155 JP 2006-4100 A International Publication No. 2018/61977 JP 2017-144989 A

In the method of Patent Document 1, in order to use the occupancy rate outside the train in real time, the load-adjusting data acquired by the load-adjusting device installed as vehicle equipment must be collected in an information processing device inside the vehicle and then transmitted to the ground equipment via a wireless line, which poses problems in terms of the costs associated with adding wireless equipment and modifying vehicle-specific equipment as well as the running costs of using the wireless line service.

Furthermore, when multiple railway operators share trains, the calculation of congestion rates based on weight as in Patent Documents 1 and 2 requires that data be provided by the railway operator that owns the train in order to use data on occupancy rates of trains owned and managed by other railway operators. This requires the development of a system for exchanging data between railway operators and operational rules between railway operators, resulting in problems with the complexity and cost of the system as well as operational complications.

In the passenger occupancy measurement system of Patent Document 3, the user judges the passenger occupancy of each vehicle by looking at the image of each vehicle, and the measurement is not automatic, and the judgment is subjective, so the passenger occupancy cannot be measured objectively. The system also discloses a function to extract features from vehicle images and automatically calculate the passenger occupancy rate, but there is a problem in that the amount of data and processing is large because features are extracted from image information.

In this respect, the system of Patent Document 4 also performs pattern recognition and passenger identification from three-dimensional shapes and image information, and the problem of large data volume and processing volume remains the same. In particular, one of the goals of measuring passenger occupancy is to grasp the situation during crowded times such as during commuting and school hours, and there are inevitably many trains, so the amount of image information captured and the amount of processing are huge, making real-time processing difficult. Furthermore, if the system is built through a network line, the amount of data to be sent and received is large, which may put strain on the line, slow down the communication speed, or cause the communication system itself to go down. In addition, issues of privacy and personal information, including portrait rights, have not been resolved. Moreover, with three-dimensional distribution data obtained by scanning in one direction from the side of the vehicle, there is a problem that passengers in areas that cannot be scanned cannot be counted in a highly crowded state, resulting in inaccurate detection of the number of passengers. Even with the method using panoramic images in Patent Document 4, there is a problem that in a highly crowded state, pattern recognition of two-dimensional shapes of passengers may result in erroneous detection of passengers and a decrease in accuracy due to false detection.

The present invention aims to solve at least some of the problems described above, and one of its objectives is to provide an automatic train congestion rate measurement system and program that can be introduced at low cost, reduces the amount of data and processing, and enables high-speed measurement while utilizing existing resources. Another objective of the present invention is to obtain a congestion rate estimation means that can estimate the congestion status of a moving train at low cost and with high accuracy even in conditions of heavy congestion, and that can be applied without complicating the system even in an operation mode in which train equipment is mutually shared between railway operators.

The present invention estimates the congestion rate based on the characteristics of the distribution of the three-dimensional shapes of subjects inside the vehicle, without identifying passengers based on their external features from image data inside the vehicle.

In order to solve the above problem, the train congestion rate measurement system of the present invention includes an imaging means capable of acquiring a distance image having distance information to a subject in pixel units of a two-dimensional image, the imaging means capturing a distance image of the train from the side in the direction of train travel, a line image generating means dividing the distance image of the train captured by the imaging means in the height direction to generate a plurality of line images, a histogram generating means generating histogram data with distance as a class from the line image generated by the line image generating means, a feature data acquiring means acquiring feature data including a height-related element, a histogram frequency-related element, and a distance-related element from the histogram data generated by the histogram generating means, and an estimation means using the feature data to estimate the congestion rate of the train.

Furthermore, in the above congestion rate measurement system, it is preferable that the feature data is two-dimensional feature data in which the frequency element is displayed at each coordinate on a plane with the height element on one axis and the distance element on the other axis.

Furthermore, in the above-mentioned congestion rate measurement system, it is preferable that the feature data acquisition means generates feature data from overall histogram data generated based on multiple distance images.

Furthermore, in the above-mentioned congestion rate measurement system, it is preferable that the congestion rate measurement system has a function of normalizing the overall histogram data by the number of frames of the distance image before acquiring the feature data.

Furthermore, in the above congestion rate measurement system, it is preferable that the congestion rate measurement system has a function of detecting the unit judgment area of the train based on the distance image or the histogram data before acquiring the feature data.

Furthermore, in the above-mentioned congestion rate measurement system, it is preferable that the histogram generation means generates the histogram data using distance information of pixels of the distance image whose distance to the subject is within a predetermined range.

Furthermore, in the above-mentioned congestion rate measurement system, it is preferable that the congestion rate measurement system has a function of detecting window areas in the distance image before acquiring the feature data.

Furthermore, in the above congestion rate measurement system, the histogram generation means may generate histogram data using distance information of pixels within the window area among the pixels of the distance image, and the estimation means may estimate the congestion rate within the window area. Furthermore, the estimation means may estimate the congestion rate of the unit judgment area of the train by tallying up the congestion rates within the window areas included in the unit judgment area of the train. Furthermore, the estimation means may tally up the congestion rates by applying different weights depending on the width, size or shape of the window area.

Furthermore, in the congestion rate measurement system, the estimation means may estimate the congestion rate by multiplying the congestion rate at the distance to the subject from the center of the train by a predetermined multiple.

Furthermore, in the above congestion rate measurement system, the estimation means may change parameters in the congestion rate calculation according to the frequency distribution in the feature data.

Furthermore, in the above congestion rate measurement system, the estimation means may correct the frequency distribution in the feature data according to the arrangement of structures within the train. Furthermore, the congestion rate measurement system may obtain information regarding the arrangement of structures within the train from feature data based on a distance image taken of a train with no passengers on board.

Furthermore, in the above congestion rate measurement system, the congestion rate measurement system may determine the vehicle type of the train and estimate the congestion rate using parameters corresponding to the vehicle type.

Furthermore, in the congestion rate measurement system, it is preferable that the estimation means estimates the congestion rate of the train using a regression model trained with the characteristic data as an independent variable and the congestion rate as a dependent variable.

The processing device of the congestion rate measurement system of the present invention also includes an image acquisition unit that acquires a distance image of the train obtained by capturing an image from the side in the direction of travel of the train using an imaging means capable of acquiring a distance image having distance information to a subject in pixel units of a two-dimensional image, a line image generation unit that divides the distance image of the train acquired by the image acquisition unit in the height direction to generate a plurality of line images, a histogram generation unit that generates histogram data with distance as a class from the line image generated by the line image generation means, and a data output unit that outputs the histogram data.

The processing device of the congestion rate measurement system of the present invention includes a data acquisition unit that acquires histogram data with distance hierarchical levels generated from a plurality of line images that are generated by dividing a distance image of the train obtained by imaging the train from the side in the direction of travel in the height direction using an imaging means capable of acquiring a distance image having distance information to a subject in pixel units of a two-dimensional image, a feature data acquisition unit that acquires feature data including elements related to height, elements related to histogram frequency, and elements related to distance from the histogram data acquired by the data acquisition unit, and an estimation unit that estimates the congestion rate of the train using the feature data.

The program of the present invention also causes a computer to realize an image acquisition function that acquires a distance image of the train obtained by capturing an image from the side in the direction of travel of the train using an imaging means capable of acquiring a distance image having distance information to a subject in pixel units of a two-dimensional image, a line image generation function that divides the distance image of the train acquired by the image acquisition unit in the height direction to generate a plurality of line images, a histogram generation function that generates histogram data with distance hierarchical from the line image generated by the line image generation means, and a data output function that outputs the histogram data.

The program of the present invention provides a computer with a data acquisition function for acquiring histogram data with distance hierarchical levels generated from a plurality of line images generated by dividing a distance image of a train obtained by imaging the train from the side in the direction of travel in a height direction using imaging means capable of acquiring a distance image having distance information to a subject in pixel units of a two-dimensional image;
The data acquisition unit realizes an image acquisition function that acquires the histogram data, a feature data acquisition function that acquires feature data including elements related to height, elements related to histogram frequency, and elements related to distance from the histogram data acquired by the data acquisition unit, and an estimation function that estimates the congestion rate of the train using the feature data.

The congestion rate measurement system, processing device, or program of the present invention can easily measure the congestion rate of a train by using an imaging means capable of capturing a two-dimensional distance image that is relatively inexpensive and commercially available, and a data conversion process that can be realized by a computer. In particular, the distance image captured by the imaging means is divided in the height direction by a line image generation means to generate line images of multiple heights, and histogram data with distance as a class is generated from each line image. The generated histogram data is aggregated to obtain feature data including elements related to height, elements related to frequency, and elements related to distance, and the congestion rate of the train is estimated using the feature data. This makes it possible to reflect the differences in spatial characteristics that occur for each height in the histogram data while ensuring them with the number of divisions, and thus makes it possible to significantly reduce the amount of data, increase the processing speed and communication speed, and improve real-time performance. In this way, the congestion rate can be easily grasped by a system that can be introduced relatively inexpensively, so that train operating companies can make efficient operation plans and improve platforms and the number of trains. In addition, by displaying the congestion rate to the user, it becomes easier to select an empty train, which leads to congestion mitigation. Furthermore, the present invention can estimate the congestion rate with high accuracy even in highly crowded or empty conditions, and can estimate the congestion rate with high accuracy even if there are differences in the structure inside the vehicle.

One embodiment of the congestion rate measurement system of the present invention Overall process flow chart Data conversion diagram for overall processing Examples of interior conditions, distance images and heat maps for seated passengers only. Examples of interior conditions, distance images and heat maps with seated and standing passengers. Examples of interior conditions, distance images and heat maps at high occupancy. System configuration diagram of one embodiment of the present invention Vehicle cross-section showing camera and passenger positions FIG. 1 is a diagram showing the configuration of an embodiment of an imaging means. 1 is a block diagram of an embodiment of a processing device; FIG. 1 is a diagram illustrating the principle of a stereo camera. FIG. 2 is a diagram showing the relationship between the shooting direction of a camera and the position and horizontal position of a subject. A diagram explaining division processing and histogram processing. FIG. 13 is a diagram for explaining an example of a unit determination region determination process; FIG. 13 is a diagram for explaining an example of a unit determination region determination process; FIG. 13 is a diagram for explaining a process for generating a group of aggregated histogram data from each unit determination region. A diagram explaining the feature data acquisition process An explanatory diagram of the processing configuration for estimating congestion rates from heat maps. Scatter plot of congestion rate (true value) and congestion rate (estimated value) An example of a flowchart of a main program in the system configuration diagram of FIG. Example of table configuration of data used in congestion rate estimation process Illustration of how to detect train ends and couplings Example of a program flowchart for calculating congestion rate An explanatory diagram of the method for calculating the estimated congestion rate for one vehicle Window width and window height detection method A diagram illustrating a method for estimating the occupancy rate from multiple images of a single window. Example of table configuration that stores vehicle type parameters used in congestion rate estimation processing Example of a flowchart for a program to obtain vehicle information Train location information diagram FIG. 1 is a diagram showing a modified example of the imaging means. Example of heat map regarding correction method for crowding rate estimation FIG. 1 is an explanatory diagram of a processing configuration for correction processing in congestion rate estimation. An illustration of the range used to estimate the congestion rate An illustration of the range used to estimate the congestion rate An illustration of the range used to estimate the congestion rate Example of vehicle interior and heat map near vehicle door Example of camera position and image capturing of the front car of a train

In the congestion rate measurement system or program of the present invention, when a moving train is observed from a side window, if the congestion rate is low, the space between passengers will be wider, and the seats and walls at the back will be visible, and conversely, if the congestion rate is high, the space between passengers will be narrower, and standing passengers will approach the seats and windows at the front, making it difficult to see the back. In order to capture such phenomena as spatial characteristics, an image is taken using an imaging means (depth camera) that can acquire distance images containing distance information (also called depth) to the subject on a pixel-by-pixel basis. This makes it possible to acquire distance images containing information on the spatial distribution of distances to the subject, which are used as a basis for judging the congestion rate. Furthermore, in the present invention, since the data volume is large when the distance image is used as is, which slows down the processing speed and communication speed, the distance image is divided in the height direction to generate multiple horizontal line-shaped divided images (hereinafter referred to as "line images"), and histogram data is generated from each line image with distance as a class. In the present invention, the generated histogram data is collected to obtain feature data including elements related to height, elements related to histogram frequency, and elements related to distance, and the congestion rate of the train is estimated using the feature data. In this way, the feature data of the present invention includes elements related to height, elements related to histogram frequency, and elements related to distance, and although the horizontal information of each distance image is discarded, the spatial distribution features of the height and distance directions can be included with a resolution corresponding to the number of divisions of the line image for the other height directions and with a resolution corresponding to the class of the histogram for the distance direction. This allows the amount of data to be significantly reduced, the processing speed and communication speed to be increased, and real-time performance to be improved. Even if the horizontal information of each distance image is discarded, the horizontal distance distribution in each image only shows a specific state in which a part of the imaging range is imaged from one side when imaging a running train, and depends on the timing of imaging. Furthermore, the information on the horizontal change of the image has little value in performing detailed analysis to estimate the congestion rate, other than information showing the window frames and coupling parts, so that the congestion rate of the train can be estimated from the feature data including other information. The feature data may be, for example, two-dimensional feature data in which the frequency element is displayed at each coordinate on a plane with the height element on one axis and the distance element on the other axis. If the frequency element is represented by a shade of color, it becomes a heat map. In the present invention, for example, the train congestion rate can be estimated using a regression model trained with the feature data as an independent variable and the congestion rate as a dependent variable, and the accuracy of the regression model can be improved by increasing the amount of data.

Below, a vehicle congestion rate estimation system according to an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are given the same reference numerals.

[Configuration of the congestion rate measurement system]
FIG. 1 shows an embodiment of a congestion rate measurement system 1 (hereinafter also referred to as a "measurement system") of the present invention. The measurement system 1 transmits a distance image F of a train captured by an imaging means 11 arranged on the side of a train 2 in a traveling direction 7 to a line image generation means 12, transmits line images F-1 to F-4 generated by the line image generation means 12 to a histogram generation means 13, transmits histogram data H-1 to H-4 generated by the histogram generation means 13 to a feature data acquisition means 14, transmits feature data C acquired by the feature data acquisition means 14 to an estimation means 15, and estimates the congestion rate of the train in the estimation means 15. The estimated congestion rate may be output to an output means 16. The congestion rate of the train 2 is an index indicating the degree of the amount of actual transported goods (people or objects) in the train 2, and includes the passenger load rate. For example, in the case of transporting people, the ratio of the actual number of passengers when the capacity of the train is set to 100% may be used as the congestion rate. In addition, as a guide to the degree of congestion, 100% means "you can sit in a seat, hold on to a strap, or hold on to a pillar near the door", 150% means "you can read a newspaper easily by unfolding it", 180% means "you can read a newspaper if you make an effort such as folding it", 200% means "your bodies are touching each other and there is a considerable sense of pressure, but you can still read a weekly magazine", and 250% means "every time the train shakes your body becomes tilted and you cannot move, even your hands".

The train 2 is a means for moving people or objects as objects, and includes electric trains, steam trains, freight cars, monorails, etc. However, the train 2 must have an object confirmation area where at least a part of the object (people or objects) can be confirmed from the outside by the imaging means 11. In the case of a train, the object confirmation area is a window (car window) 5a on the side or a window (door window) 5b on the door. The present invention is particularly preferably applied to trains that carry a large number of passengers. The present invention can also be used for trains that have multiple unit judgment areas (cars) that serve as units of areas for determining the degree of congestion, etc., such as each car of a train 2 in which multiple cars 3 are coupled by coupling parts 4, and can also be used for trains with one unit judgment area, such as a train 2 with one car.

The imaging means 11 is an imaging means capable of measuring the distance to a subject in pixel units of a two-dimensional image, and captures a two-dimensional distance image of the train 2 from the side in the traveling direction 7 of the train. A distance image is a two-dimensional image in which each pixel contains information on the distance to the subject (also called depth). A depth camera or an infrared camera can be used as the imaging means 11. The depth camera may be, for example, a TOF (Time Of Flight) type that irradiates a laser pattern of infrared light on the subject and receives the reflected light with an infrared camera to measure the distance based on the arrival time of the light, or a stereo camera type. In addition to distance images, the imaging means 11 may be capable of capturing normal two-dimensional images using visible light.

The line image generating means 12 divides the acquired distance image in the height direction to generate multiple line images F-1 to F-4, and the histogram generating means 13 generates histogram data H-1 to H-4 from each line image, reducing the amount of data and facilitating subsequent processing and transmission and reception via the network. The feature data acquiring means 14 acquires feature data including elements related to height, elements related to histogram frequency, and elements related to distance from the histogram data H-1 to H-4, and the estimation means 15 estimates the congestion rate of the train using the feature data. The specific processing in each means will be described later. As these means, an information processing device in which a line image generating program, a histogram generating program, a feature data acquiring program, and an estimation program are installed, such as a personal computer, a server, a tablet terminal, a PDA (Personal Digital Assistant), a multi-function mobile phone (smartphone, i-phone (registered trademark)), and a semiconductor integrated circuit, can be used. The device including these means has a data acquiring unit that acquires data and a data output unit that outputs processed data. The data acquisition unit may be a previous stage means connected to the device (e.g., imaging means 11 for line image generation means 12), a storage medium in which the data to be acquired is stored, a connection means for connecting a storage medium (e.g., a USB terminal, a memory slot, an optical drive), or a communication means for receiving data through a network (e.g., a wired communication means, a wireless communication means, etc.). The data output unit may be a subsequent stage means connected to the device (e.g., histogram generation means 13 for line image generation means 12), a connection means for outputting data to a storage medium (e.g., a USB terminal, a memory slot, an optical drive), or a communication means for transmitting data through a network (e.g., a wired communication means, a wireless communication means, etc.).

The output means 16 is a means for outputting the estimation results, and may be, for example, a display, projector, printer, speaker (audio output), etc., as appropriate. A display of a personal computer, tablet terminal, PDA (Personal Digital Assistant), or multi-function mobile phone (smartphone, iPhone (registered trademark)) may be used. The output means 16 may be installed on a station platform, waiting room, etc., or the estimation results may be transmitted via a network using a user's terminal device (tablet terminal, PDA, multi-function mobile phone, etc.) as output means, or the estimation results may be displayed on an Internet webpage.

The imaging means 11, the line image generating means 12, the histogram generating means 13, the feature data acquiring means 14, the estimation means 15 and the output means 16 may be realized by separate hardware, or some of the means may be realized by one hardware, or all of the means may be realized by one hardware. In particular, the imaging means 11 transmits the distance image F, which has a large amount of data, to the line image generating means 12, so it is preferable to integrate them or connect them by a dedicated line. Furthermore, the line image generating means 12 transmits the line images F-1 to F-4, which have a relatively large amount of data, which are also divided in the height direction, so it is preferable to integrate them or connect them by a dedicated line. The histogram data H-1 to H-4 have a relatively small amount of data, so they may be transmitted to the external feature data acquiring means 14 via a network. The feature data acquiring means 14 and the estimation means 15 require relatively high-performance computing power for data processing, so they may be realized by separate hardware that can be connected via a network, such as a server. In this case, by installing the imaging means 11, the line image generating means 12, and the histogram generating means 13 at multiple locations and transmitting the histogram data to a server including the feature data acquiring means 14 and the estimating means 15 via a network, it is possible to estimate the congestion rate of the same or different trains, and to collect data on the overall congestion rate. Furthermore, by training the estimating means using this collected data, it is possible to improve the accuracy of the congestion rate.

[Overall processing]
FIG. 2 is a flowchart of the overall processing in the congestion rate measurement system 1 of the present invention, and FIG. 3 is a diagram for explaining the data conversion in the overall processing of the present invention, in which (A) is a distance image F1, (B) is the arrangement of passengers when viewed from above inside the vehicle, (C) is a line image F1-c obtained by dividing the distance image in the height direction, (D) is histogram data H1-c generated from the line image F1-c, (E) is a frequency distribution (feature data) H1 obtained by aggregating the histogram data with the depth and height directions as coordinates, and (F) shows the depth-direction classes.

First, in S1, the measurement system 1 captures a two-dimensional distance image from the side of the train using the imaging means 11. Fig. 3 (A) shows a distance image F1 captured by the imaging means 11 of the interior of the train shown in Fig. 3 (B). Each pixel of the two-dimensional distance image F1 has distance information to the subject, and the distance to each subject may be indicated by, for example, pixel shading according to the class shown in Fig. 3 (F).

Next, in S2, the measurement system 1 divides the two-dimensional distance image F1 into n in the height direction (vertical direction) by the line image generating means 12 to generate line images F1-1 to F1-n. In FIG. 3(A), three line images F1-a, F1-b, and F1-c are shown as examples, but it is preferable to divide at least the entire area including the transported goods confirmation area of the distance image in the height direction. However, it is also possible to divide a part of the transported goods confirmation area, or the entire distance image in the height direction. FIG. 3(C) shows one line image F1-c extracted. The line image of the present invention is generated by dividing the distance image in the height direction. In the case of an image with H×W pixels, where H pixels in the height direction and W pixels in the width direction, the number of divisions n in the vertical direction is 2≦n≦H, and the size of the line image is H/n×W. The situation inside the train varies greatly depending on the train car, the time, and the day of the week, and is very diverse. Considering the diversity of the subject, the number of divisions is determined taking into consideration that if the number of divisions is too small, the spatial features of the transported object in the height direction may be overlooked, and if the number of divisions is too large, the data size will become large. Furthermore, the number of divisions may be changed depending on the resolution of the distance image. Although not limited to, n=4, n=8, n=16, n=32, or n=H may be used. In addition, to make the scale uniform, it is preferable to divide at equal intervals, but it is also possible to weight the division width and narrow the division width in areas where many transported objects overlap to increase the resolution of the features in the height direction, and to increase the division width in areas where there are few transported objects. In one embodiment, when a depth camera with a resolution of H×W=848×480 is used to divide into eight equal intervals, eight line images of 108×480 are generated.

Furthermore, in S3, the measurement system 1 generates histogram data H from the line image by the histogram generating means 13. In addition, in S4, the measurement system 1 may determine a unit determination area (e.g., a vehicle) from the histogram data, as necessary. FIG. 3(D) shows histogram data H1-c generated from the line image F1-c, where the horizontal axis represents the class of the distance to the subject, and the vertical axis represents the frequency of pixels. The histogram data of the present invention is obtained by counting the number of pixels at distances included in each class as a frequency, with distances of a certain width being classified as classes. Typically, the histogram is expressed as a column graph, but is not limited to those expressed as graphs, and may be data that can grasp the distribution of the frequency included in each class width, including, for example, those expressed in the form of a frequency distribution table or text information. In addition, the histogram data may be generated for one distance image, or may be generated for line images of distance images (including each frame image of a video) of multiple images (e.g., unit determination areas) captured continuously. When generating histograms for multiple frame images, the histograms of line images of the same height in each frame image may be aggregated to generate overall histogram data. The overall histogram data may be generated by a histogram generating means, or an overall histogram generating means may be provided separately. For example, the overall histogram generating means may be provided as part of the feature data acquiring means.

In addition, the measurement system 1 may normalize the histogram data in S5 as necessary. It is preferable to normalize the histogram data by the number of frames, the number of pixels, the size of the transported object confirmation area, etc. of the unit judgment area on which the data is based. For example, instead of using the first histogram data H1 based on the h1 x w1 line image and the second histogram data H2 based on the h2 x w2 line image as is, the first histogram data (e.g., H1/h1 x w1) and the second histogram data (e.g., H2/h2 x w2) whose frequencies are normalized by the number of pixels are used, and the data can be standardized, making it easy to use them mutually as comparison targets. Also, if there are 240 frame images of car 1 and 150 frame images of car 2, the overall histogram data H3 compiled based on the frame images of car 1 is normalized by the number of frames to generate normalized overall histogram data (H3/240), and the overall histogram data H4 compiled based on the frame images of car 2 is normalized by the number of frames to generate normalized overall histogram data (H4/150). This allows the data to be standardized and makes it easy to use the data mutually as a comparison target. Note that, if frame images that do not include a transported item confirmation area are excluded and only frame images that include a transported item confirmation area are used to generate overall histogram data for a unit judgment area, normalization may be performed by the number of frame images that include a transported item confirmation area in the unit judgment area, or by the total number of frame images in the unit judgment area (including the number of frame images that do not include a transported item confirmation area).

In S6, the measurement system 1 acquires feature data C from the histogram data by the feature data acquisition means 14. The feature data of the present invention includes at least an element related to height, an element related to the frequency of the histogram, and an element related to distance. For example, the feature data may be two-dimensional feature data in which the frequency element is displayed at each coordinate on a plane with the element related to height as one axis and the element related to distance as the other axis. However, this is not limited to this. FIG. 3(E) displays the feature data in a three-dimensional frequency distribution consisting of three axes, the depth direction, the height direction, and the number of pixels, but it may also be displayed as two-dimensional feature data (heat map) in which the frequency of the number of pixels is shown by the shade of the pixels, as in the feature data C of FIG. 1. Each frequency-related element is based on the frequency of each class (the count unit of the histogram, typically the number of pixels), and includes the frequency itself, the relative frequency, the cumulative frequency, the cumulative relative frequency, the normalized frequency, the weighted frequency, etc., the distance-related element is based on the distance range in each class, and includes the class, the class value, etc., and the height-related element includes the height-direction characteristics of the subject, and includes the frequency distribution divided in the height direction, etc. In addition, the feature data may be generated based on one frame image, or may be generated from overall histogram data generated based on multiple frame images (e.g., unit determination regions). Furthermore, the feature data may be normalized by the number of frames and the resolution (number of pixels) on which the data is based.

In S7, the measurement system 1 estimates the congestion rate from the feature data C by the estimation means 15. The estimation means 15 can estimate the congestion rate of the train in each car using a regression model trained with the feature data as an independent variable and the congestion rate as a dependent variable. The feature data uses the feature data in the format used during learning. For example, when using a regression model trained using feature data generated from overall histogram data based on multiple frame images of the unit determination area, feature data is similarly generated from overall histogram data based on multiple frame images of the unit determination area, and the congestion rate is estimated using the feature data and the regression model. Also, when using a regression model trained using feature data generated from histogram data based on one frame image, feature data is generated from histogram data based on one frame image, and the congestion rate is estimated using the feature data and the regression model. Furthermore, in S8, the measurement system 1 determines whether or not there is next data, for example, feature data based on another distance image or another unit determination area, and if there is next data (Yes), it returns to S7 again to estimate the congestion rate of the next data, and if there is not (No), it outputs the estimation result as necessary and ends the process. Note that S4 may be omitted if there is only one unit determination area. Furthermore, the measurement system 1 may perform a removal process in any of S2 to S5 to remove pixels outside a predetermined range.

Figures 4 to 6 are diagrams explaining the state of the interior of the train and the tendency of the feature data obtained from the distance images captured from the same, showing examples of feature data 41 generated based on distance images for the cases of only seated passengers, seated and standing passengers holding onto straps, and high congestion levels, image 42 of the interior of the train as seen from the train window, and state 43 of the interior of the train as seen from the top of the train. The feature data 41 is obtained by dividing image 42 captured by the imaging means 11 into line images, generating histogram data from the line images, and displaying the data as a heat map in which the number of pixels is represented by shading, with the depth and height directions as coordinates. Here, each coordinate consisting of the depth axis and the height axis is called a "cell." The relationship between the shading and the number of pixels of the cells in the heat map is also shown in the figures. By displaying the data as a heat map, the distribution of passengers 10 in the train can be visually grasped.

Figure 4 shows the case where only seated passengers 10a and 10b are present in the vehicle, and the pixels of the subject are distributed in the heat map 41 in the low area 41a at the front of the vehicle and in the low area 41b at the back, and the shading is correlated with the number of seated passengers in the image. In Figure 5, in addition to the seated passengers 10a and 10b, there are also standing passengers 10c and 10d, so the heat map area 41a corresponding to the seated passenger 10a is dark, as in Figure 4, but the heat map areas 41c and 41d corresponding to the standing passengers 10a and 10b near the straps are also relatively dark, and the pixels of the subject are also distributed at higher positions. However, the distribution is relatively light in the area 41e corresponding to the center of the aisle. Figure 6 shows a case where there are more standing passengers 10e than in Figure 5, and corresponds to the seated passengers 10a at the front and the standing passengers 10e overlapping near the straps and in the center of the aisle, the heat map has a wide area with a dense distribution from a low position at the front to a high position near the straps at the back. On the other hand, in Figure 6, the area 41f near the seated passengers at the back as seen from the camera 11 has a faint distribution, but this occurs because the back is not visible in the camera image due to the standing passengers at the front and the passengers standing overlapping in the center of the aisle. In this way, the feature data reflects the situation inside the car, and it is possible to estimate the congestion rate using the feature data.

In addition, since the present invention calculates the number of pixels of the subject without image recognition of passengers, some of the areas 41a and 41b contain pixels that show seats. However, when estimating the congestion rate, the pixels at the coordinates that correspond to seats on the heat map may be excluded from the calculation or may be weighted less in the calculation.

The present invention calculates the congestion rate based on the distribution of subjects captured in image data of the interior of a vehicle taken through the vehicle window. As one example, the congestion rate can be calculated by dividing the above-mentioned heat map by depth and height, assigning weights to the number of cells corresponding to the coordinate position of each cell, and aggregating the results.

Figure 7 is a system configuration diagram of an embodiment of the present invention, where Figure 7 (A) is a view from above and Figure 7 (B) is a view from the front. As shown in Figure 7 (A), the imaging means 11 and the processing device 21 are installed at the end in front of the train 2's stopping position in the traveling direction 7 of the station platform 8 where the train 2, which is made up of multiple cars 3 connected by a coupling section 4, is stopped. When the train 2 moves in the traveling direction 7, the imaging means 11 captures an image of the side of the car, which is stored in the memory of the processing device 21 and the congestion rate is estimated. The processing device 21 corresponds to the line image generating means 12, the histogram generating means 13, the feature data acquiring means 14 and the estimating means 15 in Figure 1. The imaging means 11 is attached to the platform safety fence 8a, for example, and is positioned at a distance and height such that the transported goods confirmation area (window) of the train 2 is included in the shooting range 11a. Depending on the positional relationship between the imaging means 11 and the train 2, a minimum distance L1 and a maximum distance L2 may be set so that the train 2 is included, and only pixels of distance information included in the processing target range L between the minimum distance L1 and the maximum distance L2 may be set as the target for processing. The imaging means 11 is preferably installed so that the optical axis is perpendicular to the side of the train, that is, parallel to the normal line of the side of the train. When installed perpendicularly, the distance to the side of the train is approximately uniform, making it easy to separate information about the inside and outside of the train, and improving accuracy. However, even if the optical axis is tilted, the effect of error can be reduced by machine learning, and the error may be corrected as shown in FIG. 12 described later. In FIG. 7, the imaging means 11 is installed on the platform of the station, but it may be installed next to the track beyond the platform. In addition, for example, if an imaging means is also installed on the opposite side and distance images are captured from both sides, more information for estimation can be obtained and accuracy can be improved. Although not shown, a sensor that confirms the approach of a train and starts imaging by the imaging means may be installed at the same position as the imaging means or between the imaging means and the stopping position.

The processing device 21 is installed on the platform 8 and is connected to the imaging means 11 via a wired dedicated line. Furthermore, the processing device 21 is connected to a network N, and may be connected to a train traffic control system 22, an information distribution server 23, a mobile terminal 24 held by a user 10 (including passengers) and the like via the network N. The train traffic control system 22 is a system that manages train operations based on a planned timetable, and has information such as the train number, type of train, and current location of each train, and the processing device 21 can obtain information from the train traffic control system 22 as necessary. The information distribution server 23 provides the congestion rate to the mobile terminal 24 of the user 10, a bulletin board on the station platform 8, and the like.

Figure 8 shows a state in which a seated passenger 10a and a standing passenger 10b are inside a vehicle, photographed from the side by the imaging means 11 through the vehicle window 5. Passenger 10a is seated on a seat 6 provided on the side of the vehicle, and passenger 10b is standing near the center of the vehicle. In Figure 8, the processing target range L for estimating the congestion rate is the range inside the vehicle from the front boundary L1 to the rear boundary L2. In this way, if the inside of the vehicle is set as the processing target range L, stickers and labels attached to the vehicle windows can be excluded. However, since the distance to the imaging means 11 varies due to the shaking or vibration of the vehicle, and passengers may be touching the door windows when the vehicle is full, the processing target range L may be set to include the outside of the vehicle as shown in Figure 7 (b) rather than only the inside of the vehicle.

Figure 9 is a block diagram of one embodiment of the imaging means 11, which is made up of an optical lens 100, an image sensor 101 that acquires two-dimensional data of the subject image, an image memory 102 that reads this data, a CPU 103 that controls the entire imaging means 11, a storage 104 that records the program that controls the camera and the captured images, a RAM 105 used in the execution of the program, a communication interface 106 that processes communication with an external network N, and an internal bus 107 that connects the various internal parts. The external network N includes a dedicated line for connection to the processing device 15.

Figure 10 is a block diagram of one embodiment of the processing device 21, which is made up of a CPU 110 that performs the main calculations for estimating the congestion rate in the present invention, a storage 114 that stores a program 111, parameters 112 used in the calculations, measurement data 113 such as data used in the calculations and data resulting from the calculations, a calculation unit 115 that performs product-sum calculations and the like, a RAM 116 that is used in executing the program, a communication interface 117 that processes communication with the external network N, and an internal bus 118 that connects the various internal parts.

Fig. 11 is a diagram for explaining the principle of a stereo camera, which is one of the mechanisms for recognizing a three-dimensional structure, as an embodiment of the imaging means of the present invention. As shown in Fig. 11(A), a left camera incorporating a camera lens 100a and a two-dimensional image sensor 101a and a right camera incorporating a camera lens 100b and a two-dimensional image sensor 101b are arranged at a distance L from each other, and when the interior of the vehicle is photographed by both cameras, as shown in Fig. 11(B), the subject is photographed with a shift between an image 108a captured by the left image sensor 101a and an image 108b captured by the right image sensor 101b. The images 108a and 108b in Fig. 11(B) are compared in the CPU 103 or the processing device 15 of the imaging means to detect a shift between the two images, as shown in Fig. 11(C), and parallaxes dW ₁ and dW ₂ are detected for passengers 10c and 10d, respectively. The above-mentioned parallax is converted into a distance D2 to passenger 10c and a distance _D1 to passenger 10d using the equation shown in Fig. 11C based on the distance _L between the focal points of the two camera lenses 100a and 100b and the focal length f of the cameras, and the distances are associated as distance data with each pixel of the subject captured in the two-dimensional camera image to generate a distance image (data in three-dimensional space).

Figure 12 is a diagram explaining a method for correcting errors when the optical axis 11b of the imaging means 11 is tilted. The imaging means 11 is preferably installed so that its optical axis is perpendicular to the side of the train, but since it is installed taking into consideration safety and the location and angle at which the interior of the train can be viewed through the train windows, in reality, the optical axis 11b of the imaging means 11 may be tilted from the normal to the side of the train, which is the subject. If this tilt is large, the error between the horizontal distance D from the imaging means 11 to the subject and the distance R from the lens focus to the subject in the optical axis direction becomes large. In this case, it is preferable to convert the distance R to the subject in the distance image to the horizontal distance D by the coordinate conversion shown in Figure 12, for example, from the angle θ between the normal and the optical axis and the angle φ between the passenger 10, who is the subject, and the optical axis.

[An example of obtaining feature data from multiple frame images]
13 to 17 are diagrams for explaining an embodiment of acquiring feature data from a plurality of frame images, and show an example of acquiring feature data using a series of videos captured from the side of a running train 2 from the front car to the last car. The cars 3 of the train 2 are connected to each other by couplings 4.

Figure 13 explains the division process S2 in the line image generating means 12 and the histogramming process S3 in the histogram generating means 13. Figure 7 (A) shows one frame image F1 in a moving image of a distance image, which is divided into eight equal intervals in the height direction by the line image generating means 12 to generate eight horizontally long line images F1-1 to F1-8. In this embodiment, a depth camera with a resolution of H x W = 848 x 480 is used to divide the image into eight equal intervals, generating eight line images of 108 x 480.

Figure 13 (B) shows a histogram data group H1 generated from line images F1-1 to F1-8. Each histogram data H1-1 to H1-8 included in the histogram data group H1 is generated from the line images F1-1 to F1-8 at each height, and shows the frequency distribution of distance at each height. Figure 13 (C) shows an enlarged view of histogram data H1-5 generated from the fifth line image F1-5 from the bottom. In the histogram data, the vertical axis is the frequency (number of pixels), the horizontal axis is the distance class, and the number of pixels of the distance included in each class width is the frequency. In addition, the measurement system 1 may perform a process to remove unnecessary pixel information before the division process or the histogram process. In Figure 13 (B), a range to be processed for the distance information of each pixel is set before the histogram process, and a removal process is performed to ignore pixels of distance information outside the range. Specifically, for distance information, the minimum distance L1 to be processed is set to 500 mm, the maximum distance L2 is set to 3500 mm, and the processing range L is set to 500 mm to 3500 mm (see Figures 7 and 8 for L, L1, and L2). Furthermore, the number of bins in the histogram is set to 35, the class width is set to 100 mm, the number of pixels included in each class width is counted, and histogram data is generated. Note that the processing range, number of bins, and class width are merely examples and are not limited to these.

These division and histogramming processes are performed on each frame image of a series of moving images captured by the imaging means 11 of the running train 2, and the generated histogram data is transmitted to the feature data acquisition means 14. For example, if the imaging means 11 captures images of a train at 60 fps (frames per second), and it takes about 20 seconds to capture an image of the entire 10-car train, then about 1200 frame images will be captured. A distance image (one frame image) of H x W = 848 x 480 pixels is about 800 KB (kilobytes) if 16-bit distance information is stored in each pixel, and 1200 images will be a huge amount of data of about 932 MB (megabytes). By contrast, by performing preprocessing including histogramming, the weight of the entire 10-car train can be reduced to about 1.5 MB.

[Unit Judgment Area Judgment Process]
14 and 15 are diagrams for explaining an example of the determination process of a unit determination area (vehicle). The determination process of a unit determination area (vehicle) is executed by either the preprocessing means 12 or the estimation means 13, or by a processing means provided separately for the determination process of a unit determination area (vehicle). FIG. 14(A) shows histogram data groups H1, H2, ... Hn (n is the number of frames) corresponding to each frame image, and the histogram data group Hn includes eight levels of histogram data Hn-1 to Hn-8 corresponding to eight levels of line images divided in the height direction. FIG. 14(B) shows time series frequency distribution data T1 to T8 at each height, which are collected in the order of frame numbers for each level of histogram data, and FIG. 14(C) shows an enlarged view of time series frequency distribution data T5 relating to histogram data Hn-5 at the fifth height from the bottom. The time series frequency distribution data T1 to T8 has the distance class on the vertical axis and the frame number on the horizontal axis, and the frequency (number of pixels) is displayed in shading at each coordinate. From this time series frequency distribution data, the characteristics of the time series changes can be grasped and the unit determination area can be determined.

Figure 15 (A) is a distance image F2 of the coupling part 4 of the vehicle, and Figure 15 (B) is an enlarged view of a part of the time-series frequency distribution data of Figure 14 (C). Normally, the coupling part 4 is narrower than the width of the vehicle 3 and is also lower in height, so that in the frequency distribution data of a specific height where the coupling part 4 is present, a feature that the frequency distribution at a specific distance (the distance from the imaging means to the coupling part) is high periodically appears. In the part 4a surrounded by white in Figure 15 (B), a signal with a periodically high frequency distribution is detected, and it is determined that the coupling part 4 is located at the detected frame number, and the area up to the coupling part 4 can be determined as the unit determination area (one vehicle). In this way, the boundary (coupling part) of the unit determination area may be determined, but the unit determination area may also be determined from the features detected within the unit determination area. For example, in the frequency distribution data of a height where the body of the train is normally detected and is outside the detection range of the coupling part 4, the unit determination area may be determined because a signal caused by the body of the train that is normally detected is not detected. Furthermore, a composite determination may be made from the frequency distribution data of multiple heights. FIG. 15(C) is a diagram showing the time series frequency distribution data T5 (FIG. 14(C)) for the fifth height from the bottom, with the positions of the coupling parts 4 determined by the determination process divided and the unit determination areas (vehicles) clarified. Note that this determination process is not necessary for trains with only one unit determination area.

[Normalization]
FIG. 16 is a diagram for explaining the process of generating a summary histogram data group from each unit judgment area. FIG. 16(A) is a diagram in which the orientation of the time series frequency distribution data T5 at the fifth height from the bottom in FIG. 15(C) is rotated 90 degrees to the right, and FIG. 16(B) is a summary histogram data group A5 at the fifth height from the bottom generated by aggregating histogram data in each unit judgment area. The summary histogram data group A5 includes summary histogram data A5-1 of the first car, the first vehicle, to summary histogram data A5-10 of the tenth car, the vertical axis of each data represents frequency (number of pixels), and the horizontal axis represents distance classes. Furthermore, when the imaging means is installed on the station platform, the train gradually accelerates after departure, so the number of frames that can be captured by each vehicle differs depending on the speed at which it passes in front of the imaging means. In this way, when there is a difference in the passing speed of the train, it is preferable to normalize the summary histogram data group A5 based on the number of frames. For example, the overall histogram data A5-1 of the first car is a compilation of the frequency distribution of line images at the fifth height from the bottom in the images of frame numbers 1 to 240. When further normalizing, each frequency is divided by the number of frames, 240, to obtain an average frequency distribution per frame image. The overall histogram data A5-2 of the second car is a compilation of the frequency distribution of line images at the fifth height from the bottom in the images of frame numbers 241 to 390. When further normalizing, each frequency is divided by the number of frames, 150, to obtain an average frequency distribution per frame image. By performing this normalization for all cars, the difference in the number of frames due to the passing speed can be normalized, and the accuracy of the estimation can be improved, so that normalization processing is preferable. However, normalization processing is not necessary, although the accuracy will decrease. In addition, the number of frames per car may be set to a certain range by capturing an image of a train moving at a constant speed or by changing the fps of the imaging means according to the speed of the train. Then, the process of generating a group of overall histogram data is executed for all eight levels of height.

Furthermore, for ease of computation, the frequency of the overall histogram data may be divided by the area of the distance image (848 x 480) to obtain the ratio of frequency for each distance to the image area (resolution). This normalizes each frequency by the image area.

[Feature data generation]
FIG. 17 is a diagram for explaining the process of generating feature data by the feature data acquisition means 14. FIG. 17(A) shows overall histogram data groups A1 to A8 at each of the eight height levels, and each overall histogram data group Am (m is height) includes overall histogram data Am-1 to Am-10 at each unit judgment area (cars 1 to 10), and is preferably normalized by the number of frames and/or the image area. FIG. 17(B) shows two-dimensional feature data C1 to C10 in which overall histogram data at eight height levels is combined in height order for each unit judgment area (cars 1 to 10), and FIG. 17(C) is an enlarged view of two-dimensional feature data C2 for car 2. In the two-dimensional feature data C2, the vertical axis represents height and the horizontal axis represents class, and the frequency (number of pixels) standardized by shading is displayed at each coordinate. In FIG. 17C, the vertical axis indicates the pixel range in the height direction of the line image in the distance image, and rows 742 to 848 indicate the frequency (height of the pillar) of the general histogram data A1-2 at the lowest height (first step) in the second car in shading, rows 636 to 742 indicate the frequency of the general histogram data A2-2 at the height of the second step in the second car in shading, and the following rows are similarly combined in order with the frequency of the general histogram data Am-2 at the height of the mth step in the second car in shading. Such feature data indicates the frequency distribution of distance information contained in each pixel at each height, and reflects the tendency of the congestion rate. The congestion rate of the train in each car can be estimated using a regression model trained on the two-dimensional feature data C1 to C10 with the feature data as an independent variable and the congestion rate as a dependent variable.

In this embodiment, a huge number of images must be processed in order to process a video of the entire train. For this reason, processing and analysis (for example, determination and extraction of train windows, detection of joints, etc.) for individual distance images is reduced as much as possible, and distance images that do not include train windows are used as well as distance images that include train windows. Also, instead of extracting only the train window area within the distance image, the entire two-dimensional image is processed uniformly for pixels within the processing range to increase processing speed. Then, after the line image is histogrammed to reduce the amount of data, the vehicle or joint is determined as time-series data, and general histogram data is generated for each vehicle to generate feature data. For this reason, the method of this embodiment estimates the frequency distribution of the entire vehicle, not the situation at one location at the moment when each distance image is captured, by statistically applying it. Note that processing and analysis such as determination and extraction of train windows and detection of joints may be performed for each distance image, or only distance images that include train windows may be selected, and then general histogram data may be generated for each vehicle or each train window to generate feature data.

[Learning Phase]
A regression model can be learned using the congestion rate measurement system or program of the present invention. A known congestion rate and corresponding feature data are acquired, and the feature data is used as an independent variable, and the congestion rate is used as a dependent variable to learn and construct a regression model. For example, using lightened data (histogram data, feature data) stored on the cloud, a congestion rate corresponding to the stored data may be measured and acquired by measuring an index such as a load-bearing index, and the congestion rate may be associated with the feature calculated from the stored data to learn a regression model. Furthermore, by associating information such as the day of the week, time, and weather with data and combining this with machine learning, even if the interior of the car cannot be confirmed due to blinds or condensation, it is possible to estimate the frequency distribution of the interior of the car that could not be measured by using similar information such as the day of the week, time, and weather. After constructing a regression model in the learning phase, the operation phase is started, and a distance image of an actual train is captured, and the congestion rate is estimated from the feature data. Note that even during operation, data may be continuously collected and the learning phase may be performed in parallel.

[Another embodiment of the congestion rate estimation method]
18 shows another embodiment of a processing method for calculating the congestion rate based on a heat map 41 acquired from an individual distance image. In order to remove noise in the measurement system and simplify the processing, pre-processing may be performed to group several adjacent cells 44 into one segment 45 to an extent that does not impair the features of the shading distribution of the heat map 41. The coordinates of each segment 45 can be expressed as x(i, j).

The pre-processing unit 46 executes a process of rounding multiple cells 44 into one segment 45. Since the subject area corresponding to one pixel in the camera image is proportional to the square of the ratio of the distances between two points with different distances from the camera to the subject, the pre-processing unit 46 performs a process of squaring the number of pixels in each cell as a correction, and a process of finding the average or median number of pixels in each cell in the segment as the representative value y(i, j) for the segment.

The output of the pre-processing unit is multiplied by the weighting coefficient w(i, j) corresponding to each segment in a multiplier 47 that performs a product-sum operation, and then a bias value b is added in an adder 48 that performs a product-sum operation to obtain an estimated congestion rate Z.

The weighting coefficient w(i, j) here varies depending on the camera installation position; for example, if a passenger sits in a seat closer to the camera, only part of the upper body will be captured, and if a standing passenger is standing, the number of pixels in the area of the lower body captured in the camera image will be an error factor depending on whether there is a seated passenger in front. By changing the weighting according to the coordinate position on the heat map when estimating the congestion rate, the influence of these error factors can be reduced.

Figure 19 is a scatter diagram created to explain the relationship between the estimated value of the congestion rate obtained by the above-mentioned process and the corresponding true value of the congestion rate. The true value of the congestion rate was obtained using the load-varying data described in Patent Document 1, and the estimated value was obtained and plotted by applying an embodiment of the present invention to the vehicle. Here, a congestion rate of 50% corresponds to the extent that all the seats are occupied by passengers, 100% corresponds to the extent that all passengers are seated or can hold onto a strap or a pillar near the door, and 150% corresponds to the extent that shoulders are touching and one can read a newspaper comfortably. This corresponds to the range from the state where all the seated passengers at the front and back are visible in the image of the interior of the vehicle seen from the window to the state where standing passengers at the back are visible even if the vehicle is crowded, and this is the range whose characteristics are shown in the distribution of the heat map shown as an embodiment of the present invention.

The parameter values of the weighting coefficient w(i, j) and bias b in FIG. 18 used to estimate the congestion rate are determined so that Q shown in equation (2) obtained by taking the sum of squares of the residual between m sample data and the estimated congestion rate Zk calculated by multiple regression equation (1) in which the congestion rate _Vk obtained based on the load-responsive data is used as the dependent variable and _yk (i, j) obtained by the present invention for the image data of the vehicle is used as the independent variable in the congestion rate range of 50 _% to 150%, where the characteristics appear in the heat map distribution, is minimized, and the calculation can be performed using the least squares method.

In implementing the present invention, the weighting coefficient w(i,j) and bias b are calculated by the above-mentioned method based on data previously measured experimentally, and the values are stored in the processing device 21. The congestion rate estimated by the processing device 21 may be transmitted via a network to an information distribution server in a railway information distribution system shown in Patent Document 2, for example, and used on a user's mobile terminal, etc.

[Another example of system processing]
Fig. 20 shows a flowchart of the main program in the system configuration diagram of Fig. 7, and Fig. 21 shows an example of the configuration of each data table stored in the storage 114 of the processing device 21 and written and read in the congestion rate estimation process, which stores a train-specific measurement table 240, a car-specific measurement table 241, a measured image data table 242, and a heat map table 243. After starting up, the processing device 21 sets a recording start pointer for the train-specific measurement data 240 that records the measurement data in the storage 114, and in the case where information on train location information is used (for example, variant 7), it connects to the train operation control system 22, performs initialization 200 within the processing device and starts up 210 the camera 11, and then causes multiple cameras to simultaneously take images 211 so as to obtain images at the same timing from the multiple cameras.

The processing device 21 acquires images captured by the multiple cameras 11, and executes process 201 to generate distribution data of the subjects in three-dimensional space, for example, using the stereo camera principle shown in FIG. 11. The processing device 21 continues with process 202, which detects the approach of a train, based on the results of process 201, and returns to process 201 to repeatedly acquire images from the cameras until the approach of a train is detected.

Figure 22 shows an example of a method for detecting the coupling and end of a vehicle in one embodiment of the present invention, where (A) is a diagram showing the positional relationship between the leading vehicle 3a of the train and the camera 11, and the side of the train and the camera's shooting range 31, (B) is a diagram showing the positional relationship between the coupling 4 between vehicles 3a and 3b and the camera 11, and the side of the train and the camera's shooting range 32, (C) is an image taken when the leading vehicle enters the camera's shooting range 31, (D) is an image taken when the coupling 4 enters the camera's shooting range 32, (E) is a graph 35 of the depth direction of the subject pixel at the height of the dashed line 34 in (C), and (F) is graphs 35a and 35b of the depth direction of the subject pixel at the heights of the dashed lines 34a and 34b in (D), respectively.

The approach of the leading car of a train can be detected, for example, by detecting a large change that occurs continuously in the vertical direction across the entire camera image when the background 33 of the car in the camera image in Figure 22 (C) changes to car 3a, or by using a method of measuring the distance from the camera 11 to the subject and detecting that the depth graph 35 of the pixels at the height of the dashed line 34 changes from farther away than the boundary L2 to closer than the boundary L1.

The passing of the train coupling 4 can be detected by, for example, detecting a change in image features such as the color or brightness of a vertically continuous strip in the camera image of FIG. 22 (D), or by detecting that a graph 35a of the depth direction of pixels at the height of the dashed line 34a, which includes the train window, or a graph 35b of the depth direction of pixels at the height of the dashed line 34b, which includes the train window, passes over boundary L1 from just before boundary L1 and enters the processing range between boundaries L1 and L2.

Detection of the passing of the end of the last car of the train can be achieved, for example, by detecting that the depth of the pixels of the subject is continuous beyond the boundary L2 using the method of measuring the distance from the camera 11 to the subject described above. Note that the boundaries L1 and L2 in Figure 22 are set within the interior of the car as in Figure 8. If the boundary is set outside the car as in Figure 7, the joint and the end of the car can be detected by the pixel depth instead of the boundary L1.

In process 202 of FIG. 20, if the depth direction of the subject image shown in FIG. 22(E) appears from the background 33, which is farther away than the boundary L2, to a distance closer to the boundary L2, and thus the approach of a train is detected, process 203 is started to estimate the congestion rate of a train made up of multiple cars. When using train location information (for example, variant 7), the program shown in FIG. 28 described later obtains the relevant train number and car type from the train operation control system 22 and writes them into the train-specific measurement table 240, and resets the car number counter for counting the cars to be measured.

The process of estimating train congestion rates consists of a process 201 of acquiring images from multiple cameras and generating three-dimensional spatial distribution data of the subject, a process 204 of detecting the coupling parts of the cars, a process 205 of detecting the car windows, and a process 206 of generating a heat map, which are then repeatedly executed until the end of the train is detected. Based on the data measured during this process, the congestion rate for each window image detected is estimated, and the congestion rate for each car is estimated, and the congestion rate for the entire train is calculated, and the results are transmitted.

The coupling detection process 204 may detect that the subject image shown in FIG. 22(F) is a vehicle coupling 4 by detecting a certain horizontal length in the depth direction within the processing range from boundary L1 to boundary L2, starting just before boundary L1. If detected (Yes in 204), the process executes process 207, which records the value of the car number counter for the immediately preceding vehicle, the number of window images detected in the measurement of the immediately preceding vehicle, and the index of the measurement image table in the car-specific measurement table 241 (see FIG. 21), increments the car number counter, and resets the window image detection counter that counts the window images detected for each vehicle, and prepares to measure the next vehicle.

Window image detection 205 detects whether the image contains an image of the interior of the vehicle as seen through the vehicle window. If an image of the interior of the vehicle is present (Yes in 205), a heat map is generated by tallying the distribution of pixels in the subject image by depth and height using the process shown in FIG. 3, the results are recorded in the measured image data table 242 and the heat map table 243 (see FIG. 21), and process 208 is executed to increment the window image detection counter. Whether an image of the interior of the vehicle as seen through the vehicle window is present in the image may be detected by, for example, detecting that the image contains an image of the interior of the vehicle as seen through the vehicle window in the captured image when the graph 35a in the depth direction of the pixels at the height of the dashed line 34a including the vehicle window is located within the processing target range from boundary L1 to boundary L2, as shown in graph 35a in FIG. 22(F).

The end of the train (206) can be detected by detecting the appearance of a certain length in the depth direction of the subject image from a distance before the boundary L2 to a distance farther away from the boundary L2, in the same way as detecting the leading car of the train.

In process 209, the processing device 21 reads out the train-specific measurement table 240 and the car-specific measurement table 241 from the storage 114 and transmits them to the information distribution server 23. After transmission is complete, the processing device 21 increments the write pointer of the train-specific measurement table to prepare for measuring the next train. In process 212, the information distribution server 23 receives information from the processing device 21 and can obtain estimated congestion rates for the train and each car at the measurement location and measurement time of the camera 11 for the train identified by the train number, and uses this as distribution content 213 to be provided to general users as railway information.

Fig. 23 shows a flow chart of the program for calculating the congestion rate in step 209. The flow of the program shown in Fig. 23 is as follows: in step 230, the number of cars N and a pointer to the car-specific measurement table are obtained from the train-specific measurement table 240 in Fig. 21 for data for one train. Thereafter, a loop of steps 231 to 235 is executed to calculate the congestion rate of a car designated by a counter g indicating the car number for each car, and then in step 236, the calculation results of the congestion rates for all cars are obtained from the car-specific measurement table 241, and the congestion rate Z _train of the train is calculated as the average value, and in step 237, the result is recorded in the train-specific measurement table 240, and the program ends.

Processes 231 to 235 are a loop for calculating the congestion rate for one car, in which heat maps H _{g,h for the window images detected in that car are read out sequentially from the car-specific measurement table 241 based on the number of detected window images M g and} a pointer to the heat map table 243 for each image, the calculation shown in FIG. 18 is performed to determine an estimated congestion rate for each window image, and this is written into the measured image data table 242. After calculation of the congestion rate estimates for all window images in that car has been completed, an estimated congestion rate Z car for that car is determined based on the window width recorded in the measured image data table 242 and the congestion rate estimates determined from each image, for example by the method shown in FIG. ₂₄ , and this is written into table 241.

FIG. 24 shows an example of a method for estimating the congestion rate of one vehicle from the congestion rate estimated from an image of a car window, where (A) is the side of the vehicle, and (B) shows a calculation formula for the congestion rate estimated value Z _car of one vehicle. In FIG. 24(A), the dashed square is the range 70 photographed by the camera, the gray area therein is the window range 71 in which the congestion rate in the image is estimated, the width of the window range 71 is the window width W _i , and the congestion rate estimate value Z _i obtained from each image is shown. β _i in FIG. 24(B) is a coefficient for changing the weighting according to the window width W _i , and by setting it to a small value when the window width is equal to or smaller than a certain size, it is possible to reduce the influence of the congestion rate estimate value obtained from a narrow window width such as a door window, and by setting the small value to zero, it is also possible to exclude the congestion rate estimate value obtained from the door window.

Figure 25 shows an example of a method for detecting the range of a train window in one embodiment of the present invention. Graph 36, which plots the distance from the camera to the subject on a horizontal a1-a2 cross section, shows values within the window range that are deeper than boundary L1. By detecting a point that crosses boundary L1 from the near side to the far side, the start point of the window in the horizontal direction can be detected, and by detecting a point that crosses boundary L1 from the far side to the near side, the end point of the window in the horizontal direction can be detected, and the width of the window corresponds to the window width. Similarly, graph 37, which plots a vertical b1-b2 cross section, shows values within the window range that are deeper than boundary L1, and by detecting a point that crosses boundary L1, the vertical frame of the window can be detected, and the width of the window corresponds to the window height. The detected window width and window height are recorded in the measured image data table 242.

In this embodiment, the congestion rate inside the train window is calculated from images captured by a camera, but if the window width exceeds the horizontal shooting range of the camera, or if the camera image captures only part of the train window rather than the entire window due to the timing of the camera's shooting, the congestion rate inside the train window captured in each camera image is weighted by the width of the train window captured in the camera image, making it possible to accurately calculate the congestion rate without having to stitch together the camera images to generate a digital image that captures the entire train window.

FIG. 26 shows an example of a process of taking a weighted average by the width of the car window shown in the image, where (A) is a diagram showing the entire car window with a long width, (B) is an image taken of the left side of the car window, (C) is an image taken of the center part of the car window, and (D) is an image taken of the right side of the car window. If the window widths detected in each image are W ₁ , W ₂ and W ₃ , and the congestion rates estimated from each image are Z ₁ , Z ₂ and Z ₃ , the congestion rate Z _window estimated from the entire car window shown in (A) can be obtained by the formula (E). This method, unlike the method shown in Patent Document 4 in which a panoramic image is generated by stitching together images taken continuously and then the congestion rate is estimated, contributes to simplifying and speeding up data processing. Note that when the calculation of the congestion rate estimation is changed depending on the window width, the window width recorded in the measured image data table 242 is used.

When the parameters used to calculate the congestion rate from the heat map in process 233 of Figure 23 are to be switched for each vehicle type, the vehicle type is obtained from the train-specific measurement table 240, and the parameters for the relevant vehicle type are selected and used using the parameter table 244 in which the parameters for each vehicle type are recorded. Figure 27 is an example of the parameter table 244 stored in the storage 114 of the processing device 21. The parameter table 244 has a structure that allows parameters to have different values depending on the vehicle type, and the relevant parameters are selected and used based on the vehicle type information obtained from the train operation management system 22, for example.

Figure 28 shows a flowchart of a program that accesses the train traffic control system 22. This program is called and executed by process 203 in Figure 20, and the processing device 21 executes process 220 to acquire train location information 223 from the train traffic control system 22, process 221 to extract the train number and type of train whose train position is at the installation location of the camera 11, and process 222 to write this to the train-specific measurement table 240. In Figure 28, the train traffic control system 22 is accessed each time the leading car of the train is detected by the main program, but the train traffic control system may be accessed at regular intervals asynchronously with the main program, and the latest train location information may be stored in the processing device 21 and referenced, eliminating the effect of the time required to access the train traffic control system on the processing timing of the main program.

Figure 29 is a diagram explaining train location information 223, where (A) shows the relationship between train position 80, the next train 81 to be estimated for congestion rate, and camera measurement position 82 according to the present invention, and (B) shows an example of train location information having train number, train position, and vehicle type information that identifies the train stored in the train traffic control system 22. By acquiring train location information from the traffic control system 22 at regular intervals and updating it to the latest information, the processing device 21 can know in advance the information on the vehicle type of the next train that will pass in front of the camera 11, and by setting in advance parameters for congestion rate estimation according to the vehicle type, congestion rate estimation that takes differences in vehicle structure into account can be performed.

[Modification 1]
Fig. 30 is a diagram for explaining a modified example of the imaging means 11. The imaging means in Fig. 9 has one optical lens per unit, but in this embodiment, two lenses 120, 121 and two image sensors 122, 123 are provided, and the imaging means 11 has an image memory 124 that can access digital images that can be simultaneously acquired from the two lenses.

The camera with two lenses 120, 121 shown as an example in Figure 30 was developed to acquire three-dimensional image data for autonomous driving and robots, and in addition to the optical lenses, it has a built-in program 125 for obtaining three-dimensional image data using the principles of a stereo camera, a calculation unit 126, a CPU 103, and a RAM 105, and the three-dimensional image data can be transmitted from the communication interface 106 to the line image generation means via the network N.

[Modification 2]
31 and 32 are diagrams for explaining a modification of the congestion rate estimation method of FIG. 18. In FIG. 18, the weighting coefficient w(i, j) and the bias b are fixed values set in advance, but in this embodiment, the weighting coefficient and the bias are made variable according to the data of the heat map 41. In the scatter diagram of FIG. 19, an error occurs in area 53 where the estimated value is lower than the true value of the congestion rate. This error occurs because, at a high congestion rate, the probability that passengers at the back will not be visible due to the overlap of standing passengers increases. By making the weighting coefficient and the bias variable, such an error can be corrected.

Figure 31 (A) is an example of a heat map 41 consisting of 5 x 5 segments, (B) shows the weight coefficient correction value in each segment under normal conditions, and (C) shows the weight coefficient correction value in each segment under correction. The presence or absence of correction may be determined, for example, based on the total number of pixels in a predetermined segment range in the heat map 41. In Figure 31, if the total number of pixels in area 54 of segments (3, 2), (3, 3), and (3, 5) is equal to or greater than a certain amount, it is determined that there are many passengers near the straps and near the aisles, and as shown in (C), for area 55, the number of pixels in the lower left side of the heat map where the number of pixels of the subject is reduced by standing passengers is corrected by using a weight coefficient w (i, j) multiplied by a coefficient corresponding to each i and j of the weight coefficient w (i, j), and in other cases, the coefficient correction may be performed as shown in (B). Alternatively, the bias b can be changed without changing the coefficient for the weight. For example, if the sum of the pixel counts of segments (3,2), (3,3), and (3,5) is equal to or greater than a certain value, the bias value can be increased according to the value of that sum.

Figure 32 is an example of a configuration diagram in which the weighting coefficients and biases are variable depending on the data in the heat map 41. In addition to the configuration in Figure 18, a weighting coefficient and bias value control unit 49 has been added that receives as input the values obtained by performing preprocessing 46 on each segment of the heat map, determines whether correction is required, and outputs the normal or corrected weighting coefficient w(i,j) and bias b.

[Modification 3]
Fig. 33 is a diagram for explaining a modified example for correcting the influence of the seat 6 when estimating the congestion rate. In the scatter diagram of Fig. 19, the estimated value tends to be higher than the true value of the congestion rate in area 51, but this is thought to be because there are few passengers seated on the seats and the number of pixels of the seats at the back side captured in the camera image affects the estimation of the congestion rate. Therefore, as shown in Fig. 33, the influence of the seat 6 can be corrected by setting the area 56, which excludes not only the boundaries L1 and L2 but also the segment corresponding to the seat 6b at the back side from the target of aggregation, as the range used for estimating the congestion rate.

[Modification 4]
FIG. 34 is a diagram for explaining another modified example for correcting the influence of the seat 6 when estimating the congestion rate. In the modified example 3, the area for estimating the congestion rate was obtained in advance using the design information of the vehicle, etc., but in this embodiment, the positions of the cells occupied by the objects of the structures inside the vehicle as seen from the camera and the cells that are hidden behind the structures are recorded in the processing device 21 from the data of the heat map for the camera image obtained when the vehicle is running in an empty state, so that information on the target cells or segments for estimating the congestion rate can be obtained. FIG. 34 shows an example of a heat map 41 that can be obtained when the vehicle is running in an empty state, in which the cells 44a occupied by the vehicle structures among the cells 44 in the heat map 41 are hatched, and the cells 44b that are hidden behind the vehicle structures are shown in gray. Among these, cells whose pixel count changes depending on the presence or absence of passengers are a cause of error in estimating the congestion rate, and the cell or segment including the cell where the seat 6b on the far side from the camera is located corresponds to this and is excluded from the congestion rate estimation, and the number of pixels that does not change depending on the presence or absence of passengers, for example, the luggage rack 9, etc., occurs as a fixed value and can be corrected by the bias b in the congestion rate estimation.

The present invention estimates congestion rates using camera images, and the parameters used to estimate congestion rates may vary depending on factors such as the camera installation position. However, by using heat map data acquired when the vehicle is running empty, it is possible to simplify parameter adjustments due to differences in camera installation positions when measurements are taken at multiple locations.

[Modification 5]
FIG. 35 is a diagram for explaining a further modification of the congestion rate estimation method, in which (A) shows the state inside the vehicle when the congestion rate is 200% or more, and (B) shows the heat map 41. In this embodiment, when the structure of the vehicle is symmetrical in the direction of travel, passengers need to board without large gaps at a high congestion rate, so the passengers are assumed to be arranged roughly evenly on the left and right, and calculations are simplified and a highly accurate estimated value is obtained. In modification 2, in order to correct errors that occur because the probability that passengers at the back are not visible due to the overlap of standing passengers at a high congestion rate increases, the weighting coefficient and bias are made variable according to the distribution of the heat map. However, for areas where the number of pixels of the subject is zero, such as the area 61 at the bottom left of the heat map 41 in FIG. 35(B), correction by multiplying the weighting coefficient cannot be performed. Considering that the arrangement of passengers inside the vehicle can be considered to be even on the left and right at a high congestion rate, for example, the congestion rate for an area 62 closer to the camera than the center line 60 is used as the estimated congestion rate at the back, and a highly accurate estimated value can be obtained even in a state of high congestion rate. Moreover, this can be realized by performing calculations only for area 62 in the configuration of congestion rate estimation shown in Fig. 18, and processing can be simplified. In addition, the same processing can also be realized by correcting the weighting coefficient w(i,j) of Modification 2, for example, by setting the weighting coefficient w(i,j) multiplied by y(i,j) corresponding to the segments (n,4) and (n,5) on the right side of the center in Fig. 31(A) to 2, and setting the weighting coefficient w(i,j) multiplied by y(i,j) corresponding to the segments (n,1) and (n,2) on the left side of the center to 0. This coefficient switching may be performed, for example, using the value of area 56 in Fig. 31 described in Modification 2 as a condition.

[Modification 6]
FIG. 36 is a diagram for explaining congestion rate estimation based on the door window 5b of the vehicle door, where (A) is a view from the side of the vehicle when there is a passenger 10e standing close to the door window 5b, (B) shows the situation inside the vehicle, and (C) shows an example of a heat map 41 corresponding to the door window 5b. The door window 5b of the vehicle is also subject to congestion rate estimation, and even if the seat is vacant, there may be a passenger standing close to the door of the vehicle, and the visibility inside the vehicle may be restricted by the passenger 10e standing close to the narrow door window, resulting in a heat map distribution different from that of the vehicle window near the seat. In the heat map of FIG. 36(C), a high pixel distribution is shown in the entire area 65 from a low position to a high position on the front side corresponding to the passenger 10e standing close to the door window. On the other hand, in other areas, the distribution of the image of the subject is small due to the passenger 10b standing close to the narrow door window. For this reason, as shown in Fig. 24(B), in the calculation formula of the congestion rate estimate value _Zcar for one vehicle, the weighting coefficient _βi by the window width W _i is set to a small value when the window width is equal to or smaller than a certain size, thereby making it possible to reduce the influence of the door window in the congestion rate estimate for one vehicle. Note that the congestion rate estimate value obtained from the door window can also be excluded by setting the coefficient _βi of the door window to zero.

[Modification 7]
This embodiment relates to a method for improving the estimation accuracy of the congestion rate even in cases where various vehicle types are operated when parameters related to the structure of a vehicle are used to estimate the congestion rate. For example, the measurement range considering the range of vehicle structures such as seats captured in a camera image in Modification 3, and the weighting of the congestion rate estimate value obtained from a narrow window such as a door window in Modification 6, etc., are changed depending on the structure of the vehicle. In order to switch the parameters used when estimating the congestion rate of a vehicle in the present invention based on information on the vehicle type corresponding to the vehicle structure, it is possible to realize this by acquiring information on the train position and its vehicle type from the train operation management system 22, storing parameter sets for multiple vehicle types in advance in the processing device 21, and using the parameters for the corresponding vehicle type for estimating the congestion rate every time a train is detected.

For example, the processing device 21 accesses the traffic control system 22 at regular intervals, acquires train location information as shown in FIG. 29(B), and updates it to the latest information, thereby being able to determine in advance the information on the type of train that will pass in front of the camera next, and can perform congestion rate estimation taking into account differences in vehicle structure by setting parameters for congestion rate estimation according to the type of train with reference to the parameter table 244 (FIG. 27). Also, as shown in FIG. 37(A), a camera 25 other than the imaging means 11 that captures distance images of the side of the train is installed in a direction 25a that captures the front of the leading vehicle 3a of the train, and the image recognition program implemented in the processing device 21 determines the type of train from the image of the front of the train in FIG. 37(B) captured by the camera 25, and can set parameters for congestion rate estimation according to the type of train with reference to the parameter table 244 (FIG. 27), thereby performing congestion rate estimation taking into account differences in vehicle structure. The image recognition program may determine the type of train from the design of the front of the train, or may determine the train number 91 on the front of the train and/or the characters of the vehicle model 92 corresponding to the type of train.

In the above variants 1 to 7, heat maps based on individual distance images have been described, but feature data (heat maps: e.g., FIG. 17) generated from overall histogram data based on multiple distance images can also be used. In addition, the configurations described in this specification are merely examples, and they can be combined with each other, and some configurations in one embodiment can be applied to another embodiment.

Reference Signs List 1: Congestion rate measurement system 2: Train 3: Car 4: Coupler 5, 5a, 5b: Car window 11: Imaging means 12: Line image generation means 13: Histogram generation means 14: Feature data acquisition means 15: Estimation means 16: Output means

Claims (20)

An imaging means capable of acquiring a distance image having distance information to a subject in pixel units of a two-dimensional image, the imaging means capturing a distance image of the train from a side in the train's traveling direction;
a line image generating means for generating a plurality of line images by dividing the distance image of the train captured by the imaging means in a height direction;
a histogram generating means for generating histogram data with distance as a class from the line image generated by the line image generating means;
a feature data acquisition means for acquiring feature data including a height element, a histogram frequency element, and a distance element from the histogram data generated by the histogram generation means;
an estimation means for estimating a congestion rate of the train using the feature data;
A congestion rate measurement system including: The congestion rate measurement system according to claim 1, wherein the feature data is two-dimensional feature data in which the frequency element is displayed at each coordinate on a plane with the height element on one axis and the distance element on the other axis. The congestion rate measurement system according to claim 1 or 2, wherein the feature data acquisition means generates feature data from overall histogram data generated based on multiple distance images. The crowding rate measurement system according to claim 3, which has a function of normalizing the overall histogram data by the number of frames of the distance image before acquiring the feature data. The congestion rate measurement system according to any one of claims 1 to 4, wherein the congestion rate measurement system has a function of detecting a unit judgment area of the train based on the distance image or the histogram data before acquiring the feature data. The crowding rate measurement system according to any one of claims 1 to 5, wherein the histogram generation means generates histogram data using distance information of pixels of the distance image whose distance to the subject is within a predetermined range. The congestion rate measurement system according to any one of claims 1 to 6, wherein the congestion rate measurement system has a function of detecting a window area in the distance image before acquiring the feature data. the histogram generating means generates histogram data using distance information of pixels within the window region among the pixels of the distance image;
The congestion rate measuring system according to claim 7 , wherein the estimation means estimates the congestion rate within the window area. The congestion rate measurement system according to claim 8, wherein the estimation means estimates the congestion rate of the unit judgment area of the train by tallying up the congestion rates within the window areas included in the unit judgment area of the train. The congestion rate measurement system according to claim 9, characterized in that the estimation means performs aggregation by weighting differently depending on the width, size or shape of the window area. The congestion rate measurement system according to any one of claims 1 to 10, wherein the estimation means estimates the congestion rate by multiplying the congestion rate at the distance to the subject from the center of the train by a predetermined multiple. The congestion rate measurement system according to any one of claims 1 to 11, wherein the estimation means changes parameters in the congestion rate calculation according to the frequency distribution in the feature data. The congestion rate measurement system according to any one of claims 1 to 12, characterized in that the estimation means corrects the frequency distribution in the feature data according to the arrangement of structures within the train. The congestion rate measurement system according to claim 13, wherein the congestion rate measurement system obtains information about the layout of structures within the train from feature data based on distance images taken of a train with no passengers on board. The congestion rate measurement system according to any one of claims 1 to 14, wherein the congestion rate measurement system determines the train's vehicle type and estimates the congestion rate using parameters corresponding to the train type. The congestion rate measurement system according to any one of claims 1 to 15, wherein the estimation means estimates the congestion rate of the train using a regression model trained with the feature data as an independent variable and the congestion rate as a dependent variable. an image acquisition unit that acquires a distance image of the train obtained by capturing an image of the train from a side in the traveling direction of the train using an imaging means that can acquire a distance image having distance information to a subject in pixel units of a two-dimensional image;
a line image generating unit that generates a plurality of line images by dividing the distance image of the train acquired by the image acquiring unit in a height direction;
a histogram generating unit for generating histogram data with distance as a class from the line image generated by the line image generating unit ;
a data output unit that outputs the histogram data;
A processing device for a congestion rate measurement system including: a data acquisition unit that acquires histogram data with distances as classes generated from a plurality of line images that are generated by dividing a distance image of the train obtained by imaging from the side in the traveling direction of the train using an imaging means that can acquire a distance image having distance information to a subject in pixel units of a two-dimensional image; and
a feature data acquisition unit that acquires feature data including a height element, a histogram frequency element, and a distance element from the histogram data acquired by the data acquisition unit;
an estimation unit that estimates a congestion rate of the train using the feature data;
A processing device for a congestion rate measurement system including: On the computer,
an image acquisition function for acquiring a distance image of the train obtained by capturing an image from the side in the direction of travel of the train using an imaging means capable of acquiring a distance image having distance information to a subject in pixel units of a two-dimensional image;
a line image generation function that generates a plurality of line images by dividing the distance image of the train acquired by the image acquisition function in a height direction;
a histogram generating function for generating histogram data with distance as a scale from the line image generated by the line image generating function ;
a data output function for outputting the histogram data;
A program to achieve this. On the computer,
a data acquisition function that acquires histogram data with distance hierarchical classes generated from a plurality of line images that are generated by dividing a distance image of the train obtained by imaging the train from the side in the direction of travel using an imaging means that can acquire a distance image having distance information to a subject in pixel units of a two-dimensional image ; and
a feature data acquisition function for acquiring feature data including a height element, a histogram frequency element, and a distance element from the histogram data acquired by the data acquisition function;
an estimation function for estimating a congestion rate of the train using the feature data;
A program to achieve this.