JP7303384B2 - Passenger number counting system and passenger number counting device - Google Patents

Description

The present invention relates to a passenger number counting system and a passenger number counting device.

In Japan, while the population is concentrating in urban areas, the population is declining in rural areas, and the maintenance of public transportation such as buses and trains has become a major social issue. Under such circumstances, there is a demand for further efficiency improvement in operation in order to operate bus companies and public transportation.

In order to improve the efficiency of maintenance and operation of public transportation such as buses and trains, it is first necessary to accurately grasp the usage of public transportation.
As one of technical means for achieving such an object, there is a technique for estimating the number of people staying in public places such as airports and train stations and determining the degree of congestion.
For example, WO2017/122258 (Patent Literature 1) describes, “A system for visualizing the degree of congestion in an entire facility using surveillance cameras. By deriving the size (number of people), the congestion situation within the range of the camera image is estimated.In addition, the flow of people (the amount of movement of people) is calculated and modeled based on the number of people passing by, the size of the crowd, and the speed of movement. The model is constructed by selecting the distribution function and its parameters using a statistical model selection method such as BIC (Bayesian Information Criteria). Congestion conditions are plotted on the map, and the congestion status of the area can be grasped immediately.”

WO2017/122258

In the above-mentioned Patent Document 1, in a commercial facility such as an airport or a train station, by analyzing an image from a camera exclusively installed so that the number of people staying at the commercial facility can be easily counted, congestion of the commercial facility is analyzed. Estimating the degree is described.

However, in Patent Document 1, application to buses and trains is not assumed, and a dedicated camera is required so that the number of people can be easily counted. For this reason, when applied to public transportation such as buses and trains, it is necessary to introduce new cameras, and the installation of these cameras requires a lot of work, resulting in high costs. Therefore, it is difficult to introduce, especially for local public transportation with limited budget, and there is a demand for a technology that can grasp the usage situation at a lower cost.

Therefore, the present invention uses images acquired by sensors such as cameras installed in public transportation such as buses and trains to accurately count at least the number of people getting on or off the public transportation. The purpose is to provide a means to grasp the usage status of public transportation at low cost.

In order to solve the above-mentioned problems, one typical passenger number counting device of the present invention includes a reception unit for receiving an image from a photographing device capable of photographing a vehicle entrance and exit, and a counting unit that counts the number of people getting on and off the vehicle by applying a predetermined image processing means, and outputs the counting result.

According to the present invention, by accurately counting the number of people getting on and off the vehicle using images acquired by existing sensors such as security cameras already installed in vehicles such as buses and trains, public transportation It is possible to provide a means for grasping the usage status of the at low cost.
Problems, configurations, and effects other than those described above will be clarified by the following description of the mode for carrying out the invention.

FIG. 1 is a diagram showing an example of the hardware configuration of a passenger number counting system according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a passenger number counting system according to an embodiment of the present invention. FIG. 3 is a block diagram of a computer system for implementing embodiments of the present invention. FIG. 4 is a diagram showing an example of a software configuration for implementing the embodiment of the present invention. FIG. 5 is a diagram showing the flow of the method for counting the number of boarding and alighting passengers according to the embodiment of the present invention. FIG. 6 is a diagram showing an example of analysis video according to the embodiment of the present invention. FIG. 7 is a diagram showing another example of analysis video according to the embodiment of the present invention. FIG. 8 is a diagram showing an example of counting results generated by the passenger number counting means according to the embodiment of the present invention.

(Outline of the present invention)
As described above, in order to efficiently maintain public transportation systems such as buses and trains as the population continues to decline, there is a demand for means for accurately grasping the usage status of public transportation systems.
Therefore, the present invention analyzes images of boarding/alighting gates, boarding gates, or alighting gates acquired from existing cameras installed inside buses and trains. To provide a means for automatically counting the number of people who More specifically, by analyzing the image of the entrance/exit acquired from the camera using, for example, an object detection means, an object determined to be a human being is depicted on the image. By counting the number of times that the determination line is crossed, erroneous recognition can be avoided, and the number of people getting on and off the vehicle, or the number of people getting on or off the vehicle can be accurately grasped.
In the passenger number counting means according to the present invention, since an existing camera already installed in the vehicle is used, there is no need to purchase and install a new camera, and costs can be reduced. In addition, by performing cloud-type information management and using so-called big data analysis, for example, by analyzing the usage status of transportation modes operated by different local governments, etc., we can better understand the overall usage status of a specific route. can be accurately grasped.

A conventional example and an embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited by this embodiment. Moreover, in the description of the drawings, the same parts are denoted by the same reference numerals.

First, a passenger number counting system according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG.

FIG. 1 is a diagram showing an example of the hardware configuration of a passenger number counting system 100 according to an embodiment of the present invention. As shown in FIG. 1, the passenger number counting system 100 includes client terminals 105A, 105B, 105C, a network 110, and a passenger number counting device 125. FIG. Client terminals 105A, 105B, 105C and passenger number counting device 125 are connected via network 110 .

The client terminals 105A, 105B, and 105C upload images of a boarding/alighting gate of a vehicle such as a bus or a train, or a boarding/alighting gate, for example, to a dedicated upload site (upload site 224 shown in FIG. 2, which will be described later). , to the passenger number counting device 125 to request passenger number counting processing. For example, if the vehicle is a bus operated by a municipality, the client terminals 105A, 105B, and 105C may be terminals used by the manager of the bus operator.
In addition, the client terminals 105A, 105B, and 105C may be dedicated terminals for uploading images acquired by separate cameras, and the acquired images are directly transmitted in real time via the network 110 to the number of passengers counting device. 125 may be a camera.
Although FIG. 1 shows a configuration including three client terminals 105A, 105B, and 105C as an example, the present invention is not limited to this, and the number of client terminals may be, for example, only one, or four. There may be more than one, and they may be selected as appropriate.

As described above, the passenger number counting device 125 performs image analysis processing on the analysis video to be subjected to the passenger number counting process, thereby determining the number of passengers representing the number of people getting on the vehicle and the number of people getting off the vehicle. It is a device that provides a function to determine the number of people getting off.
More specifically, passenger number counting device 125 includes reception section 130 , determination section 135 , counting section 140 , and storage section 155 .

The reception unit 130 is a functional unit that receives an image from a photographing device such as a camera installed to photograph a boarding/alighting gate of a vehicle, a boarding gate, or an alighting gate. For example, the reception unit 130 may acquire the analysis video from any of the client terminals 105A, 105B, and 105C via a dedicated upload site. The reception unit 130 also transfers the input analysis video to the determination unit 135 .

The determination unit 135 is a functional unit that determines a video segment corresponding to a stop of the vehicle from the analysis video received from the reception unit 130 . For example, the determination unit 135 analyzes the analysis video using existing object detection means, and determines a frame sequence including an event corresponding to the vehicle being stopped for a predetermined time or longer as an analysis video segment. Identify and extract the relevant video segment. Here, the determination unit 135 may extract a plurality of video segments by analyzing the analysis video from the beginning to the end using this object detection means.
In this way, the determination unit 135 uses the object detection means to analyze the video received from the reception unit 130 to identify the video when the vehicle is stopped, and identify the video when the passenger is likely to get on and off. Segments can be determined. By performing the process of counting the number of boarding and alighting passengers, which will be described later, on the video segment extracted here, the number of boarding and alighting passengers, the number of passengers, and the number of alighters can be counted more efficiently than when analyzing the entire video. .

The counting unit 140 counts the number of people getting on and off the vehicle, or the number of people getting on and/or getting off the vehicle in the video for analysis by performing a predetermined image processing means on the video segment received from the determination unit 135, Output the counting result. For example, the counting unit 140 determines the position on the image of the judgment line for counting the number of passengers (near the boarding/alighting gate, or the boarding/alighting gate, etc.) based on the angle of view of the received analysis image. After the determination line is drawn at the position, the number of people getting on and off the vehicle or the number of people getting on and/or getting off the vehicle may be tallied by counting the number of times an object determined to be a human passes through the determination line. In addition, the counting unit 140 performs this processing on each of the video segments determined by the determination unit 135, thereby calculating the number of passengers getting on and off the vehicle or the number of passengers and/or getting off the vehicle for each stop at which the vehicle stops. The number of people can be counted.

The storage unit 155 is a storage unit for storing various data used by the functional units described above. For example, the storage unit 155 may temporarily store the analysis video received by the reception unit 130, the video segment extracted from the analysis video by the determination unit 135, and the like. The storage unit 155 here may be any storage medium such as flash memory, hard disk drive, or the like.

Next, an example of a passenger number counting system according to an embodiment of the present invention will be described with reference to FIG.

FIG. 2 is a diagram showing an example of providing a passenger counting system 100 according to an embodiment of the present invention to a bus operator. In the following, a case where the passenger count system 100 according to the embodiment of the present invention is provided to a bus operator and the target vehicle for counting processing is a bus will be described as an example, but the present invention is not limited to this. The passenger number counting means according to the embodiment of the present invention may be applied to arbitrary vehicles such as automobiles, trains, monorails, airplanes and ships.

As shown in FIG. 2 , the passenger number counting system 100 according to the embodiment of the present invention may include a bus operator side 210 and a system manager side 220 . Also, the bus operator side 210 is, for example, a side managed by a company or a local government that manages the bus business, and as shown in FIG. may contain.

The bus 212 is, for example, a bus operated by a private company, a route bus operator, a local government, or the like, and may be an arbitrary bus such as a community bus or an obsolete alternative bus. Bus 212 may also include small buses, medium buses, large buses, and the like.

As shown in FIG. 2, bus 212 includes camera 213 . This camera 213 may be, for example, an existing camera such as a security camera that has already been installed, or may be a dedicated camera. is desirable. The camera 213 is configured to photograph the boarding/alighting gate of the bus 212, or the boarding/alighting gate of the bus 212, so as to acquire images of passengers getting on and off the bus. Therefore, it is desirable that the camera 213 be installed at an angle where the bus boarding/alighting gate, boarding gate, or alighting gate can be well captured.
Note that the number of cameras 213 may be arbitrarily selected depending on the size of the bus 212, the number of boarding gates, boarding gates, or alighting gates. Preferably equal to the number of mouths or exits. For example, in one embodiment, if the bus 212 has two entrances, an entrance dedicated to boarding and an exit dedicated to getting off, the bus 212 may include two cameras 213 that photograph these two entrances. .

The recorded video medium 214 is, for example, a storage medium for storing video captured by the camera 213, and may be any storage medium such as an SD card, USB memory, HDD, SSD, flash memory, or the like.

The upload PC 216 is a computing device for uploading the video captured by the camera 213 of the bus 212 and stored in the video recording medium 214 to the upload site 224 provided by the system administrator side 220 . The upload PC 216 may be any computing device such as, for example, a desktop computer, notebook computer, tablet, smart phone, dedicated terminal, or the like. The upload PC 216 corresponds to, for example, the client terminal shown in FIG.
The upload PC 216 may upload the video captured by the camera 213 of the bus 212 and stored in the recording video medium 214 via a wireless LAN such as Wi-Fi or via a wired LAN. good.

The system administrator side 220 is a side that provides and manages, for example, a function of counting the number of passengers getting on and off a vehicle, the number of passengers getting on, or the number of people getting off the vehicle. , passenger counter 125 , and CSV file 228 .

The upload site 224 , the passenger number counting device 125 and the CSV file 228 on the system administrator side 220 may be implemented in a predetermined cloud environment 230 . The cloud environment 230 here provides, for example, scalable computing resources as a service on a network. More specifically, the cloud environment 230 provides an abstraction between computing resources and their underlying technical architecture (e.g., servers, storage, networks) such that shared computing resources are available to users and administrators. A service that can be provided quickly without intervention.

The system management PC 222 is a computing device for managing the passenger number counting device 125 . The system management PC 222, for example, checks the video uploaded from the upload PC 216 via the upload site 224, adjusts the settings of the passenger number counting device 125, and performs the counting by the passenger number counting device 125. It provides a function to check the CSV file showing the results.
Also, the system management PC 222, like the upload PC 216, may be any computing device such as a desktop computer, a notebook computer, a tablet, a smart phone, a dedicated terminal, or the like.

The uploading site 224 is used for uploading to the cloud environment 230 the video captured by the camera 213 of the bus 212 and uploaded via the uploading PC 216 to be processed by the passenger counting device. Website.

The passenger number counting device 125 performs image analysis processing on the analysis video to be subjected to the passenger number counting process, which is uploaded via the upload site 224, so that each bus stop at which the bus 212 stops is displayed. This device provides a function of determining the number of passengers getting on the bus 212 and the number of people getting off the bus 212 .
The configuration of the passenger number counting device 125 substantially corresponds to the passenger number counting device 125 shown in FIG. 1, for example. Therefore, description here is omitted.

The CSV file 228 is data indicating the results of the number of passengers and/or the number of alighters determined by the boarding counting process by the boarding/alighting number counting device 125 . This CSV file 228 may be sent to the bus operator 210 or forwarded to another function for further analysis.

An example of the passenger number counting system 100 described above will be described. First, the camera 213 installed inside the bus 212 for photographing the boarding/alighting entrance, the boarding entrance, or the alighting entrance acquires images of passengers getting on and off the bus 212, and saves them in the video recording media 214. . For example, when the bus 212 returns to the garage, the bus driver or the manager of the bus operator side 210 uses the upload PC 216 to upload the video stored in the recorded video medium 214 to the system manager side 220. Upload to the upload site 224 .
The uploading site 224 that has received this video transfers the received video to the passenger number counting device 125 . The passenger number counting device 125 performs predetermined image analysis processing on the analysis video using the above-described functional units such as the determination unit and the counting unit, so that at each stop where the bus 212 stops, The number of passengers getting on the bus 212 and the number of people getting off the bus 212 are determined. The results of the number of passengers and/or the number of passengers getting off at each bus stop determined here are output to the CSV file 228 . The output CSV file may then be sent to the bus operator side 210 and forwarded to another function for further analysis.

In the above description, a configuration in which the video captured by the camera 213 is stored in the recording video medium 214 and transmitted to the system administrator side 220 via the upload PC 216 has been described as an example, but the present invention is limited to this. not. For example, the camera 213 is connected to the Internet, and a configuration is also possible in which the captured video is directly uploaded to an upload site in real time. In this case, since the recording video medium 214 and the uploading PC 216 are not required, there is an advantage that the manual uploading of the video can be saved and the counting result of the passenger number counting device 125 can be obtained in real time.
In the above, the case of analyzing the information acquired from one bus operator was shown as an example, but the present invention is not limited to this, and in one embodiment, the system administrator side 220 can analyze information acquired from the same area or route. By collectively analyzing the data obtained from multiple public transportation systems operating in the area, it is possible to grasp not only the usage status of one public transportation system, but also the usage status of the target area and the entire route.

By using the passenger count counting system 100 configured as described above, it is possible to calculate the number of passengers getting on and off the vehicle or the number of passengers using the images acquired by existing sensors such as security cameras already installed in vehicles such as buses and trains. By accurately counting the number of people or getting off, it is possible to understand the performance of the public transportation system.

Referring now to Figure 3, a computer system 300 for implementing embodiments of the present disclosure will be described. The mechanisms and apparatus of various embodiments disclosed herein may be applied to any suitable computing system. The major components of computer system 300 include one or more processors 302 , memory 304 , terminal interfaces 312 , storage interfaces 314 , I/O (input/output) device interfaces 316 , and network interfaces 318 . These components may be interconnected via memory bus 306 , I/O bus 308 , bus interface unit 309 and I/O bus interface unit 310 .

Computer system 300 may include one or more general-purpose programmable central processing units (CPUs) 302A and 302B, collectively referred to as processors 302. As shown in FIG. In some embodiments, computer system 300 may include multiple processors, and in other embodiments, computer system 300 may be a single CPU system. Each processor 302 executes instructions stored in memory 304 and may include an on-board cache.

In some embodiments, memory 304 may include random access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing data and programs. Memory 304 may store all or part of the programs, modules, and data structures that implement the functions described herein. For example, memory 304 may store passenger counting application 350 . In some embodiments, the passenger count application 350 may include instructions or descriptions that perform the functions described below on the processor 302 .

In some embodiments, the passenger counting application 350 may include semiconductor devices, chips, logic gates, circuits, circuit cards, and/or other physical hardware instead of or in addition to processor-based systems. It may also be implemented in hardware via a hardware device. In some embodiments, the passenger count application 350 may include data other than instructions or descriptions. In some embodiments, a camera, sensor, or other data input device (not shown) may be provided in direct communication with bus interface unit 309, processor 302, or other hardware of computer system 300. .

Computer system 300 may include bus interface unit 309 that provides communication between processor 302 , memory 304 , display system 324 , and I/O bus interface unit 310 . I/O bus interface unit 310 may be coupled to I/O bus 308 for transferring data to and from various I/O units. I/O bus interface unit 310 connects via I/O bus 308 to a plurality of I/O interface units 312, 314, 316, also known as I/O processors (IOPs) or I/O adapters (IOAs); and 318.

Display system 324 may include a display controller, display memory, or both. The display controller can provide video, audio, or both data to display device 326 . Computer system 300 may also include devices, such as one or more sensors, configured to collect data and provide such data to processor 302 .

For example, the computer system 300 may include a biometric sensor that collects heart rate data, stress level data, etc., an environmental sensor that collects humidity data, temperature data, pressure data, etc., and a motion sensor that collects acceleration data, motion data, etc. may include Other types of sensors can also be used. The display system 324 may be connected to a display device 326 such as a single display screen, television, tablet, or handheld device.

The I/O interface unit provides the ability to communicate with various storage or I/O devices. For example, the terminal interface unit 312 may be used for user output devices such as video displays, speaker televisions, etc., and user input devices such as keyboards, mice, keypads, touch pads, trackballs, buttons, light pens, or other pointing devices. Such user I/O devices 320 can be attached. A user inputs input data and instructions to the user I/O device 320 and the computer system 300 by operating the user input device using the user interface, and receives output data from the computer system 300. good too. The user interface may be displayed on a display device, played by a speaker, or printed via a printer, for example, via user I/O device 320 .

Storage interface 314 connects to one or more disk drives or direct access storage devices 322 (typically magnetic disk drive storage devices, but arrays of disk drives or other storage devices configured to appear as a single disk drive). ) can be attached. In some embodiments, storage device 322 may be implemented as any secondary storage device. The contents of memory 304 may be stored in storage device 322 and read from storage device 322 as needed. I/O device interface 316 may provide an interface to other I/O devices such as printers, fax machines, and the like. Network interface 318 may provide a communication pathway to allow computer system 300 and other devices to communicate with each other. This communication path may be, for example, network 330 .

In some embodiments, computer system 300 is a device that receives requests from other computer systems (clients) that do not have a direct user interface, such as multi-user mainframe computer systems, single-user systems, or server computers. There may be. In other embodiments, computer system 300 may be a desktop computer, handheld computer, laptop, tablet computer, pocket computer, phone, smart phone, or any other suitable electronic device.

Next, with reference to FIG. 4, a software configuration for implementing the functions of the passenger number counting device according to the embodiment of the present invention will be described.

FIG. 4 is a diagram showing an example of a software configuration 400 for implementing the functions of the passenger number counting device according to the embodiment of the present invention. As shown in FIG. 4, the software configuration 400 includes an image input module 402, a log output module 404, a decision line setting module 406, a passenger number counting module 408, an image processing module 410, various libraries 412, a skeleton detection module 414, a deep layer Includes learning module 416 , programming language 418 and OS 420 .
The software modules included in the software configuration 400 shown in FIG. 4 are implemented by the functional units of the passenger number counting device 125 shown in FIG. It constitutes the part.

The video input module 402 is a software module for inputting video for analysis acquired by a client terminal such as a camera mounted on a vehicle. The image input module 402 may, for example, convert the received analysis image into a format accepted by the passenger number counting module 408, which will be described later.

The log output module 404 is a software module for outputting the counting result of the passenger number counting process by the passenger number counting module 408, which will be described later, to a file such as CSV, for example.

The determination line setting module 406 is a software module for determining the position of the determination line for counting the number of passengers on the image and drawing the determination line at the determined position. The judgment line setting module 406 may determine the position of the judgment line based on the angle of view of the analysis video, for example, for each camera.

The passenger number counting module 408 is a software module that analyzes the analysis image received from the image input module 402, counts the number of passengers getting on and off the vehicle, or the number of people getting on or off the vehicle shown in the analysis image, and outputs the counting result. is. For example, as described above, the passenger count module 408 may output counts indicating the number of passengers boarding and/or exiting each stop at which the vehicle stops.

The video processing module 410 analyzes the analysis video received by the video input module 402 by moving object detection processing, etc., using existing object detection means based on object definitions and activity definitions defined in various libraries 412, for example. Then, a frame sequence containing an event corresponding to the vehicle being stopped for a predetermined period of time or longer is specified as a video segment for analysis, and after extracting the video segment, these video segments are processed by a passenger counting module. 408. The moving object detection process here is a process of tracking the position of a moving area in a video between frames to obtain a movement trajectory.
In addition, the video processing module 410 detects an object corresponding to a human in the extracted video segment using a skeleton detection module 414 such as tf pose estimation or open pose, or a deep learning module 416, and detects the detected human posture. You may also estimate the posture and the like.

The video processing methods implemented by video processing module 410 according to embodiments of the invention are not limited to skeleton detection module 414 and deep learning module 416 shown in FIG. Various image processing methods such as detection, object classification, etc. can be used.
For example, when the video captured by the camera installed in the vehicle is shot from above, the head and skeleton of the passenger are detected using arc detection, local feature detection, and optical flow calculation. , it is desirable to determine objects that correspond to humans.

In general, arc detection is performed by extracting the contour of each video segment using, for example, an edge filter, performing a three-dimensional Hough transform on the coordinates, and voting. It is what is done. In addition to the Hough transform, methods such as edgelet features that take into account the spatial relationship of feature points and integrate or thin them out to obtain arc segments and fit them to an elliptic equation using the least-squares method. can be used.

Local feature detection detects feature points (interest points) by edge detection, Harris corner detection, or the like, and also calculates local feature amounts such as HOG (Histograms of Oriented Gradients) for each video segment. Local features can be broadly divided into those extracted from the periphery of non-uniform interest points and those extracted (uniformly) based on points on a grid (dense sampling). New features have been proposed one after another, and the local feature detection of this example can be applied to any of them. Local Binary Pattern (LBP), Haar wavelet, Gabor wavelet, texton feature, color histogram, Extended HOG, HOF (Histograms of Change Point), image moments (covariance matrix), EOH (Edge of Orientation Histograms), Edgelet, Shapelet, Harris-affine, Hessian-Affine, SURF (Speeded Up Robust Features), Principal Curvature-Based Region, IBR (intensity-extrema-based Regions), MSER (Maximally Stable Extremal Regions), FAST (Features from Accelerated Segment Test) , KAZE (wind), GLOH, binary features (BRIEF, BRISK, ORB, FREAK, CARD, BinBoost), etc. are known.

Optical flow calculation tracks feature points output from local feature detection between frames to calculate optical flow. At this time, a KLT (Kanade-Lucas-Tomasi) Tracker or a GunnarFarneback method may be applied using the original video as well. Only one (eg, median) may be selected for each grid of . Further, optical flow-based feature amounts (HOF (Histograms of Optical Flow), MBH (Motion Boundary Histogram), etc.) may be further calculated from the calculated optical flow.

It is also possible to perform face detection processing and detect a human face in the video for analysis. Generally, face detection detects a face appearing at an arbitrary location in an image based on feature amounts detected by local feature detection processing. In general face detection, Hear-like (JointHaar-like) features are used to search for a partial image whose features are most likely to be a face. A classifier represented by a linear combination of individual Haar-like features is optimized by learning by Adaboost et al. When a partial image with more than a predetermined degree of face-likeness is found, it can be easily obtained from the partial image itself, the position and size, the probability of being a face, the low-dimensional feature amount corresponding to the partial image, and the Haar-like features. Outputs the positional relationship information (face direction, etc.) of the face parts to be used.

Furthermore, in the present invention, an object classification process may be performed in order to determine whether an object getting on or off a vehicle is a person. The object classification process is a process of identifying an object in an image based on the moving trajectory from the moving object detection and the predefined features of the object (such as a wheelchair).

The software modules described above are constructed in a predetermined program language such as Python, and may be configured to operate on a predetermined OS 420 such as CentroOS.

Next, referring to FIG. 5, a method for counting the number of passengers getting on and off according to an embodiment of the present invention will be described.

FIG. 5 is a diagram showing the flow of a passenger count method 500 according to an embodiment of the present invention. The passenger number counting method 500 shown in FIG. 5 may be implemented by the functional unit of the passenger number counting device shown in FIG. 1, for example.
By executing the passenger count method 500, it is possible to accurately count the number of passengers getting on and off the vehicle, or the number of people getting on and off the vehicle, and to grasp the usage status of the vehicle.

First, in step S510, the reception unit (for example, the reception unit 130 shown in FIG. 1) receives analysis images acquired by a client terminal such as a camera mounted on a vehicle. As described above, this analysis image includes, for example, images of passengers getting on and off the vehicle. In addition, the analysis image received here may be, for example, an image acquired by one camera installed in one vehicle, or may be acquired by a plurality of cameras installed in one vehicle. It may be a video captured by a camera on a different vehicle.

Next, in step S520, the determination unit (for example, the determination unit 135 shown in FIG. 1) analyzes the analysis image using the existing object detection means to determine whether the vehicle has been stopped for a predetermined time or more. A frame sequence containing an event corresponding to the event is specified as a video segment for analysis, and the video segment is extracted. Also, here, a plurality of video segments may be extracted by analyzing the analysis video from beginning to end using this object detection means. By performing the below-described process of counting the number of boarding and alighting passengers on the video segments extracted here, the number of boarding and alighting passengers can be counted more efficiently than when the entire video is analyzed.

Next, in step S530, the counting unit (for example, the counting unit 140 shown in FIG. 1) determines the position of the determination line for counting the number of passengers on the image based on the angle of view of the video segment received from the determination unit 135. (Near the boarding/alighting gate, or the boarding/alighting gate, etc.) is determined, and the determination line is drawn at the determined position. The judgment line here is a line on the image for judging whether or not the passenger has boarded or alighted from the vehicle. Further, when determining the position of the judgment line, the counting unit sets the first direction to be the "boarding direction" with reference to the judgment line, and sets the second direction substantially opposite to the first direction. is the "getting off direction". Here, the boarding direction is the direction in which the passenger should travel when getting on the vehicle, and the exiting direction is the direction in which the passenger should be traveling when getting off the vehicle. For example, the boarding direction is the direction corresponding to the inside of the vehicle, and the exiting direction is the direction corresponding to the outside of the vehicle.
If there are video segments acquired by a plurality of cameras, the counting unit determines the position of the judgment line and the boarding direction and exit direction for each camera, and draws the judgment line on the image. good.

Next, in step S540, the counting unit analyzes the video segment extracted by the determining unit using existing object detection means to detect objects corresponding to humans in the video segment. Thereafter, the counting unit adds 1 to the count of the number of passengers when an object determined to be a human passes through the determination line along the direction determined as the boarding direction. Also, when an object determined to be a human passes through the determination line along the direction determined to be the get-off direction, the counting unit subtracts -1 from the count of the number of passengers. Alternatively, the counting unit may count separately the number of passengers and the number of passengers getting off.
Also, the counting unit uses time information such as time stamps embedded in the video segments determined by the determining unit and route information including information about stops at which the vehicle stops, to determine each video segment as route information. By repeating the above-described counting process for each video segment after associating with the designated stop, the counting result for each video segment can be associated with the stop on the route. As a result, it is possible to grasp the number of passengers and the number of passengers getting off at each stop.

By executing the passenger number counting method 500 described above, it is possible to accurately count the number of passengers getting on and off the vehicle, the number of people getting on and off the vehicle, and the usage status of the vehicle.

Next, an example of an analysis image according to an embodiment of the present invention will be described with reference to FIGS. 6 and 7. FIG.

FIG. 6 is a diagram showing an example of an analysis image 600 according to an embodiment of the invention. As mentioned above, the analysis image 600 shown in FIG. It may be a video to be shot.

As described above, the counting unit (for example, the counting unit 140 shown in FIG. 1) according to the embodiment of the present invention, based on the angle of view of the analysis video 600, on the image of the determination line 610 for counting the number of passengers position (near the boarding/alighting gate, or a boarding/alighting gate, etc.), and the determination line 610 is drawn at the determined position. Here, in order to determine the position of the judgment line 610 on the image, the judging unit uses a neural network trained based on past video information to determine the position of the passenger in the analysis video 600 when boarding and alighting. A position where the probability satisfies a predetermined probability criterion may be determined as the position of the decision line 610 .
In one embodiment, the counting unit accepts an administrator's instruction specifying the position of the determination line 610, for example, input via a system management PC (for example, the system management PC 222 shown in FIG. 2), The position of decision line 610 may be determined based on this indication.

After the position of the decision line 610 is determined, the counting unit draws the decision line 610 on the analysis image 600 at the determined position. After that, the counting section can count the number of people getting on and off the vehicle, the number of people getting on, or the number of people getting off using this decision line 610 .

FIG. 7 is a diagram showing an example of an analysis image 700 according to an embodiment of the invention. Similar to the analysis image 600 shown in FIG. 6, the analysis image 700 shown in FIG. Alternatively, it may be a video image of the exit 705 .

As described above, the counting unit (for example, the counting unit 140 shown in FIG. 1) according to the embodiment of the present invention, based on the angle of view of the analysis image 700, on the image of the determination line 710 for counting the number of passengers (near the boarding/alighting gate, boarding/alighting gate, etc.) is determined, and the decision line 710 is drawn at the determined position. After that, the counting section determines the getting-on direction 712 and the getting-off direction 714 based on the determination line 710 . For example, as shown in FIG. 7, the boarding direction 712 is, for example, the direction corresponding to the inside of the vehicle, and the exiting direction is, for example, the direction corresponding to the outside of the vehicle.

After that, the counting unit detects an object corresponding to a person, for example, using existing object detection means. Next, when the object determined to be a human passes through the determination line 710 along the direction determined as the boarding direction 712, the counting unit adds 1 to the count of the number of passengers, and the object determined to be a human is When the determination line 710 is passed along the direction determined as the getting off direction 714, -1 is subtracted from the count of the number of getting off. Alternatively, the counting unit may count separately the number of passengers and the number of passengers getting off.

Next, with reference to FIG. 8, the counting result generated by the passenger number counting means according to the embodiment of the present invention will be described.

FIG. 8 is a diagram showing an example of counting results 800 generated by the passenger number counting means according to the embodiment of the present invention. As shown in FIG. 8 , counting result 800 includes stops 812 , number of passengers 814 , and number of passengers 816 getting off.

As described above, according to the passenger number counting means according to the embodiment of the present invention, after determining the video segment corresponding to the stop of the vehicle from the analysis video, it is embedded in each video segment. By comparing time information, such as timestamps, with route information, including information about stops at which the vehicle stops, it is possible to determine which stop on the vehicle's route each video segment corresponds to. Therefore, after associating each video segment with a stop on the route of the vehicle, by repeating the above-described counting process for each video segment, the counting result for each video segment can be associated with the stop on the route. .
As a result, as shown in FIG. 8, it is possible to generate a counting result 800 indicating the number of passengers 814 and the number of passengers 816 getting off at each stop 812 . This counting result 800 may be output to, for example, a CSV file or the like, and transferred to any specified destination such as a municipality, a company, or an organization that manages the operation of vehicles.
Note that FIG. 8 shows a case where the counting result 800 is expressed as a table as an example, but the present invention is not limited to this, and the counting result 800 may be expressed as a graph or the like.

According to the passenger number counting means of the present invention as described above, the number of boarding and alighting of vehicles at each stop is calculated using images acquired by existing sensors such as security cameras already installed in vehicles such as buses and trains. The number of people can be accurately counted for each time period. As a result, it is possible to accurately grasp the usage status of public transportation at low cost, and to obtain information for considering efficient and effective operation of public transportation without the need for new capital investment. becomes possible.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present invention.

100: Passenger number counting system 105A, B, C: Client terminal, 110: Network, 125: Passenger number counting device, 130: Reception unit, 135: Judging unit, 140: Counting unit, 155: Storage unit, 210 : Bus operator side 212: Bus 213: Camera 214: Recording video medium 216: PC for uploading 220: System administrator side 222: PC for system management 224: Site for uploading 228: CSV file , 230: Cloud environment

Claims (10)

a photographing device installed so as to be able to photograph at least the entrance or exit of the vehicle;
a passenger counting device for counting at least the number of passengers boarding or exiting the vehicle;
A passenger number counting system having
The passenger number counting device
a reception unit that receives an image from the imaging device;
a counting unit that counts at least the number of people boarding or getting off the vehicle by subjecting the video to predetermined image processing means, and outputs a count result;
including
The counting unit
Based on the installation location of the photographing device and the angle of view of the image, the position of the determination line for counting the number of passengers is determined on the image, and after the determination line is drawn at the determined position, the object is determined to be a human being. By counting the number of times that passes through the judgment line, the number of passengers getting on and off or the number of passengers and / or the number of passengers getting off are counted,
In order to determine the position of the judgment line on the image, a neural network trained based on past video information is used to determine a position in the video that satisfies a predetermined probability standard for the probability that passengers will pass when getting on and off. determined as the position of the judgment line,
A passenger number counting system characterized by: The passenger number counting device
further comprising a determination unit for determining a video segment from the video corresponding to a stop of the vehicle;
The counting unit
counting the number of passengers and the number of people getting off in each video segment, and outputting the number of passengers and the number of people getting off for each video segment as the counting result;
The passenger number counting system according to claim 1, characterized by: The passenger number counting device
counting each of the video segments based on route information indicating a route traveled by the vehicle , time information embedded in each of the video segments , and route information including information about stops at which the vehicle stops; Correlate the results with the stops on the route specified in the route information ,
The passenger number counting system according to claim 2, characterized by: The counting unit
In the video segment corresponding to the stopping of the vehicle,
determining a first object corresponding to a human;
adding the number of passengers when the first object passes the determination line along the first direction;
subtracting the number of passengers when the first object passes through the decision line along a second direction that is substantially opposite to the first direction;
The passenger number counting system according to claim 1 , characterized by: The vehicle is
is selected from automobiles, buses, trains, monorails, ships, and airplanes;
The passenger number counting system according to any one of claims 1 to 4 , characterized in that: A passenger number counting device,
a reception unit for receiving an image from a photographing device installed so as to photograph at least an entrance or exit of a vehicle;
a counting unit that counts at least the number of people boarding or getting off the vehicle by subjecting the video to predetermined image processing means, and outputs a count result;
including
The counting unit
Based on the installation location of the photographing device and the angle of view of the image, the position of the determination line for counting the number of passengers is determined on the image, and after the determination line is drawn at the determined position, the object is determined to be a human being. By counting the number of times that passes through the judgment line, the number of passengers getting on and off or the number of passengers and / or the number of passengers getting off are counted,
In order to determine the position of the judgment line on the image, a neural network trained based on past video information is used to determine a position in the video that satisfies a predetermined probability standard for the probability that passengers will pass when getting on and off. determined as the position of the judgment line,
Passenger counting device. The passenger number counting device
further comprising a determination unit for determining a video segment from the video corresponding to a stop of the vehicle;
The counting unit
counting the number of passengers and the number of people getting off in each video segment, and outputting the number of passengers and the number of people getting off for each video segment as the counting result;
The passenger number counting device according to claim 6 , characterized in that: The passenger number counting device
The counting result of each video segment based on the route information indicating the route along which the vehicle travels, the time information embedded in each of the video segments , and the route information including information about stops at which the vehicle stops. to the stops on the route specified in the route information ,
The passenger number counting device according to claim 7 , characterized in that: The counting unit
In the video segment corresponding to the stopping of the vehicle,
determining a first object corresponding to a human;
adding the number of passengers when the first object passes the determination line along the first direction;
subtracting the number of passengers when the first object passes through the decision line along a second direction that is substantially opposite to the first direction;
The passenger number counting device according to claim 6 , characterized in that: The vehicle is
is selected from automobiles, buses, trains, monorails, ships, and airplanes;
The passenger number counting device according to any one of claims 6 to 9 , characterized in that:

Images