JP6492599B2 - Lighting state determination device, lighting state determination method, and lighting state determination program - Google Patents

Description

  The present invention relates to a technique for determining whether a side lamp attached to a side surface of a moving body is in a lighting state or a light-off state.

  Conventionally, in order to ensure the safety of passengers at the station platform, it is performed to detect the open / closed state of a vehicle door (door) of a train stopped at the station platform.

  For example, Patent Document 1 describes a technique for processing a frame image obtained by imaging a side surface of a train vehicle and detecting a lighting state of a vehicle side light that is turned on or off in conjunction with opening and closing of a train door. Yes. This patent document 1 extracts a high-luminance area in a frame image and identifies the color of the extracted high-luminance area, so that the lit vehicle side lamp is a door closing display lamp or an emergency lamp. It is the structure which determines. The door closing indicator lights up when the door is open and turns off when the door is closed. Further, the emergency light is turned on in an emergency (for example, when notifying an electrical error, an air conditioner failure, an emergency alarm device notification from the inside of the vehicle, etc.), and is turned off in a normal state.

  As described in Patent Document 1, the door closing indicator lamp and the emergency lamp have different lighting colors. The lighting color of the door closing indicator in a general train vehicle is red, but the lighting color of the emergency light varies depending on the type of train vehicle, such as orange, yellow, and green.

Japanese Patent No. 5506537

  However, since Patent Document 1 is configured to determine that the door closing indicator lamp is lit if the high luminance area in the frame image is the lighting color of the door closing indicator lamp, disturbance light (sunlight, In some cases, a portion irradiated with the illumination light of the station platform is extracted as a high-intensity region (a region where the vehicle side lamp is lit). And if the color of this extracted high-intensity area is close to the lighting color of the door closing indicator lamp, it may be erroneously determined that the door closing indicator lamp is lit.

  In addition, when the color of the high-intensity area extracted by irradiating disturbance light is not close to the lighting color of the door closing indicator lamp, it may be erroneously determined that the emergency lamp is lit.

  An object of the present invention is to provide a technique for accurately determining whether a side lamp attached to a side surface of a moving body is in a lighting state or a light-off state.

  In order to achieve the above object, the lighting state determination device of the present invention is configured as follows.

  The lighting state determination device according to the present invention determines whether the side lamp attached to the side surface of the moving body is in a lighting state or in a light-off state. Here, the moving body is, for example, a train vehicle. In addition, the side light is a door closing indicator that lights up when the door is open and goes off when the door is closed, or notifies in the event of an emergency (for example, an electrical error, an air conditioner failure, an emergency alarm from the vehicle, etc.) It is an emergency light that lights up when it is off and goes off during normal times.

  A frame image obtained by imaging the side surface of the moving body is input to the frame image input unit. That is, the side lamp attached to the side surface of the moving body is captured in the frame image input to the frame image input unit.

  The reference frame image acquisition unit is a frame image input to the frame image input unit, and the moving body being imaged is in a stopped state, and the side lamp attached to the side surface of the moving body is in an off state. A frame image is acquired as a reference frame image. Further, the processing target frame image acquisition unit acquires a frame image input to the frame image input unit and captured after the reference frame image acquired by the reference frame image acquisition unit as a processing target frame image. . Further, the luminance difference image generation unit generates a luminance difference image between the reference frame image and the processing target frame image.

  If the reference frame image is a frame image in which the side lamp is turned off and the processing target frame image is a frame image in which the side lamp is turned on, in the luminance difference image generated by the luminance difference image generation unit, A region where the side lamps in the lighting state are imaged appears as a region where a certain luminance difference has occurred.

  Then, the state determination unit extracts a region where a predetermined luminance difference is generated as a candidate region in the luminance difference image generated by the luminance difference image generation unit, and the extracted sidelight is in the candidate region extracted here. It is determined whether or not is captured.

  The candidate area extracted by the state determination unit is not only the area where the side lamp in the lighting state is imaged, but also the disturbance light (sunlight, A region where the intensity of the illumination light is somewhat different is also included. For example, if the luminance of the region on the reference frame image corresponding to the extracted candidate region is higher than the luminance of the region on the processing target frame image corresponding to the extracted candidate region, the state determination unit It determines with the side lamp which is a lighting state not being imaged. That is, it is possible to accurately determine whether the side lamp attached to the side surface of the moving body is in the on state or in the off state.

  The frame image input to the frame image input unit may be a gray scale image or a color image. When the frame image input to the frame image input unit is a color image, the luminance difference image generation unit converts the reference frame image acquired by the reference frame image acquisition unit into a grayscale reference grayscale image, and performs processing. What is necessary is just to set it as the structure which has the gray scale conversion function which converts the process target frame image which the target frame image acquisition part acquired into the gray scale process target gray scale image. In this case, the luminance difference image generation unit may generate a difference image between the reference grayscale image converted into the grayscale by the grayscale conversion function and the processing target grayscale image as the luminance difference image.

  In addition, the gray scale conversion function has an effect on the luminance value of a gray scale image obtained by converting a specific color in advance when converting a color image to a gray scale image. It should be larger than the influence on the luminance value. This specific color is preferably matched with the lighting color of the side lamp for determining the lighting state. In this way, when the side lamp is in the lit state, the luminance difference in the area where the side lamp is imaged increases in the grayscale luminance difference image, so the side lamp in the lit state is imaged. Occurrence of a situation where a region is not extracted as a candidate region is suppressed. On the other hand, in the grayscale luminance difference image, the luminance difference in the area irradiated with sunlight or illumination light can be suppressed to be small, so the area irradiated with sunlight or illumination light is a candidate. It is possible to suppress unnecessary extraction as a region.

  Moreover, it is good also as a structure provided with the stop detection part which detects having changed into the stop state stopped from the moving state which the moving body is moving. In this case, the reference frame image acquisition unit is a frame image input to the frame image input unit, and is captured at a timing when the stop detection unit detects that the moving body has changed from the moving state to the stopped state. What is necessary is just to acquire a frame image as a reference | standard frame image.

  For example, when the moving body is a train, the train opens the door of the vehicle after stopping at the station platform. Therefore, in the frame image whose timing at which it is detected that the train has stopped at the station platform is the imaging timing, the door closing indicator lamp which is a vehicle side lamp (side lamp in the present invention) attached to the side surface of the vehicle is turned off. It is an image of the state that is being performed. For this reason, it is possible to appropriately acquire the reference frame image in which the moving body is in the stopped state and the side lamp attached to the side surface of the moving body is in the off state.

  The state determination unit may have a resizing function for resizing the size of the extracted candidate area according to the diffusion of light in the candidate area. In this case, the state determination unit may determine whether or not a side lamp that is in a lighting state is captured in the resized area obtained by resizing the extracted candidate area using the resize function.

For example, the state determination unit may determine that the side lamp that is in the lighting state is not captured in the resize area when any of the following conditions (1) to (3) is not satisfied.
(1) The size of the resized area is not within a predetermined size range.
(2) In the resizing area, the ratio of pixels that are determined to have a predetermined luminance difference in the luminance difference image generated by the luminance difference image generation unit is not greater than a predetermined ratio.
(3) The average luminance value of the area on the reference frame image to which the resized area corresponds exceeds a predetermined luminance value.
In addition, the state determination unit may determine that the side lamp that is in the lighting state is not captured in the resized area when the plurality of conditions (1) to (3) are satisfied at the same time.

  Moreover, it has a registration part which registers the area | region which the state determination part determined that the side lamp attached to the side surface of the moving body is a lighting state as a lighting area, and a state determination part is for every lighting area which the registration part registered. Further, it may be configured to determine whether or not the side lamp located in the lighting area is extinguished, and further, the registration unit may delete the registration of the area determined to be off by the state determination unit.

In this case, if the state determination unit, the extracted candidate region is extracted candidate regions that overlap the lighting region is registered by the registration unit, for example, the following (1), one of the conditions (2) When satisfied, it is determined that the side lamp located in the lighting area is turned off.

(1) For a lighting region registered by the registration unit, pixels located in the lighting region on the luminance difference image generated by the luminance difference image generation unit at the time of registration of the lighting region, and corresponding pixels on the reference frame image Are pixels located in the candidate area on the luminance difference image generated this time by the luminance difference image generation unit with respect to the number of pixels that have a predetermined luminance difference between and the corresponding difference on the reference frame image The ratio of the number of pixels having a predetermined luminance difference with the pixels is not a certain value or more.
(2) There is an area similar to the candidate area from which the luminance value is extracted in the vicinity of the lighting area on the reference frame image.
Moreover, you may make it a structure which a state determination part determines with the side lamp located in the lighting area | region not extinguishing, when satisfy | filling the conditions of said (1) and (2) simultaneously.

In addition, when the state determination unit does not extract the candidate region that overlaps the lighting region registered by the registration unit, for example, when any of the following conditions (1) to (3) is satisfied, It is determined that the side lamp located in the lighting area is not turned off.

(1) The lighting area on the previous processing target frame image is compared with the lighting area on the current processing target frame image, and the number of pixels whose luminance value is reduced in the current processing target frame image is determined in advance. The set number has not been reached.
(2) There is no region whose luminance value is similar to the lighting region on the reference frame image in the vicinity of the lighting region on the current processing target frame image.
(3) The lighting area is not registered if the side lamp is lit for a predetermined number of frames.

  Moreover, a state determination part is good also as a structure which determines with the side lamp located in the lighting area | region not having extinguished, when satisfy | filling the said (1)-(3) several conditions simultaneously.

  According to the present invention, it is possible to accurately determine whether the vehicle side lamp attached to the side surface of the train vehicle is in the on state or in the off state.

It is a block diagram which shows the structure of the principal part of an image processing apparatus. It is a figure which shows the frame image imaged by the imaging device. It is a flowchart which shows operation | movement of an image processing apparatus. It is a flowchart which shows a stop detection process. It is a flowchart which shows a state determination process. It is a flowchart which shows a vehicle side light detection pre-process. 7A and 7B are diagrams for explaining the mask processing. It is a figure which shows a 1st state determination process. 9A and 9B are diagrams for explaining the resizing process. It is a figure explaining the search range which searches a similar area | region. It is a figure which shows a 2nd state determination process. It is a flowchart which shows the state determination process concerning another example.

  Embodiments of the present invention will be described below.

  FIG. 1 is a block diagram illustrating a configuration of a main part of the image processing apparatus. This image processing device 1 corresponds to a lighting state determination device according to the present invention. The image processing apparatus 1 includes a control unit 2, an image input unit 3, an image processing unit 4, and an input / output unit 5. The image processing apparatus 1 determines whether the vehicle side light (in this example, the door closing indicator light) attached to the side surface of the train that is stopped at the station platform is in the on state or in the off state. Then, the determination result is output to the host device. In this example, the train corresponds to the moving body referred to in the present invention. The image processing apparatus 1 can use an information processing apparatus such as a personal computer having the above-described configuration as hardware. This information processing apparatus (image processing apparatus 1) executes the lighting state determination method according to the present invention. The lighting state determination program according to the present invention is installed in the information processing apparatus (image processing apparatus 1).

  The control unit 2 controls the operation of each part of the main body of the image processing apparatus 1. In addition, the control unit 2 includes a storage unit such as a memory that stores data related to various parameters used for controlling the operation of the main body, processing data generated during the operation, and the like.

  A frame image captured by the imaging device 10 is input to the image input unit 3. The imaging device 10 is a video camera that captures a moving image in an imaging area. The frame rate of the imaging device 10 is several tens of frames / sec (for example, 30 frames / sec). The imaging device 10 according to this example captures a moving image in an imaging area in color.

  The imaging area of the imaging device 10 will be described with reference to FIG. FIG. 2 is a diagram illustrating a frame image captured by the imaging apparatus. The imaging device 10 has an angle at which the side surface of the train that stops at the station platform is within the imaging area. The imaging device 10 is attached to a pole or the like standing on the station platform. The imaging device 10 is attached with an angle in a direction in which a vanishing point of a train that is a moving body is generated, not perpendicular to the moving direction (traveling direction) of the train. As is well known, the vanishing point of the moving body is a point where the moving body gathers in the moving direction in a plane image with a sense of perspective (for example, a frame image captured by the imaging device 10). In the frame image shown in FIG. 2, the vanishing point of a train that is a moving body is located outside the frame image.

  Note that the imaging device 10 may be attached on the captured frame image at an angle at which the vanishing point of the train that is a moving object is imaged, or on the captured frame image as shown in FIG. The vanishing point of a certain train may be attached at an angle at which no image is taken.

In general stations, an imaging device for monitoring is installed for safety monitoring of passengers at the station platform. This imaging device for monitoring is for the purpose of monitoring an object such as a passenger or baggage falling from the station platform into the track, and is installed at an angle so that the end of the station platform is within the imaging area. Therefore, this monitoring imaging apparatus can also be used as the imaging apparatus 10 described above. That is, the imaging device 10 that inputs a frame image to the image input unit 3 may be newly installed at a station platform, or an existing monitoring imaging device installed for safety monitoring of passengers at the platform. May be used.

  The image processing unit 4 processes the frame image input to the image input unit 3, and the door closing indicator lamp attached to the side surface of the train vehicle captured in the frame image is on or off. It is determined whether it is in a state. Details of the process for determining the state of the door-closed indicator lamp (whether it is turned on or off) (details of image processing in the image processing unit 4) will be described later. The door closing indicator lamp corresponds to the side lamp in the present invention. In this example, the lighting color of the door closed display is red. The door closing indicator light is assigned when one or more doors of a train vehicle are assigned, and when all assigned doors are closed, the lights are turned off, and at least one of the assigned doors is open. When lit.

  The input / output unit 5 performs input / output with a host device (not shown). The input / output unit 5 outputs, for example, a determination result obtained by determining the state of the door closing indicator lamp to the host device.

  The host device is a guidance device that broadcasts a warning message to passengers staying at the station platform (for example, a message such as “A train departs from line 0. Please go down to the white line”). It may be a movable fence control device that opens and closes a movable fence (so-called home door) shown in the above-mentioned Patent Document 1 or the like, or may be another type of device. Good. Further, the image processing device 1 may be configured to notify a plurality of higher-level devices (for example, the above-described guidance device and movable fence control device) of the determination result obtained by determining the state of the train door closing indicator lamp.

  Hereinafter, the operation of the image processing apparatus 1 will be described.

  FIG. 3 is a flowchart showing the operation of the image processing apparatus.

  The image processing apparatus 1 performs a stop detection process for detecting whether a train has stopped at the station platform (s1). Details of this stop detection process will be described later. The image processing apparatus 1 repeats the stop detection process for s1 until it is detected that the train has stopped at the station platform. When the image processing apparatus 1 detects that the train has stopped at the station platform in the stop detection process for s1, the image processing apparatus 1 ends the stop detection process for s1, and the door close indicator lamp that is the vehicle side light is lit. A state determination process is performed to determine whether it is in a state or in a light-off state (s2). Details of this state determination processing will be described later. The image processing apparatus 1 repeats the state determination process for s2 until the train departs from the station platform. When detecting that the train departs from the station platform in the state determination process for s2, the image processing apparatus 1 ends the state determination process for s2, returns to s1, and repeats the above-described process.

  The stop detection process concerning s1 is demonstrated referring FIG. FIG. 4 is a flowchart showing the stop detection process. The image processing apparatus 1 according to this example performs the stop detection process in the image processing unit 4. In this example, the image processing unit 4 detects the approach, stop, departure, etc. of the train at the station platform by calculating the kinetic energy of the train in the vanishing point direction (hereinafter referred to as “disappearing energy”). A technique for detecting the approach, stop, departure, etc. of a train at a station platform based on lost energy is disclosed in Japanese Patent Application Laid-Open No. 2014-59649 relating to a previous patent application filed by the applicant of the present application. Let's briefly explain.

  As described above, the imaging device 10 inputs the captured frame image to the image input unit 3. The image processing unit 4 extracts a processing target frame image to be processed this time from the frame images input to the image input unit 3, and calculates a train lost energy VE in the processing target frame image (s11). The image processing unit 4 selects processing target frame images in time series. The image processing unit 4 sequentially extracts the frame images input to the image input unit 3 every n frames as processing target frame images. n is an integer of 1 or more.

  The image processing unit 4 sets a plurality of measurement points (not shown) in advance on the frame image (for example, the frame image shown in FIG. 2) input to the image input unit 3. Each measurement point is a point set at a position on the frame image input to the image input unit 3 (not a point set for a moving object imaged in the frame image). Each measurement point is set at a position where a train entering the station platform is imaged (a position where the train is not imaged, such as a position where the platform floor is imaged). .)

  The image processing unit 4 detects the imaging position in the current processing target frame image for the image captured at each measurement point of the previous processing target frame image. This imaging position can be detected by pattern matching using the previous processing target frame image and the current processing target frame image. The image processing unit 4 detects, for each measurement point, a vector that reaches the position detected by the above-described processing from the measurement point as a movement vector of the measurement point.

  The image processing unit 4 calculates the lost energy at each measurement point based on the movement vector at each measurement point. The vanishing energy at a measurement point can be calculated by using a unit vector in the vanishing point direction preset for the measurement point and a depth correction coefficient. The depth correction coefficient is a coefficient for correcting the magnitude of the movement vector of the measurement point according to the distance between the measurement point and the vanishing point. The vanishing energy at the measurement point is energy corresponding to the component in the vanishing point direction of the movement vector detected for the measurement point. The image processing unit 4 calculates the sum of the lost energy calculated for each measurement point as the lost energy VE in the current processing target frame image.

  The vanishing energy VE calculated here corresponds to the magnitude of kinetic energy of a moving body such as a train moving in the vanishing point direction, and increases as the kinetic energy increases. Therefore, the vanishing energy VE increases as the moving speed of a moving body such as a train moving in the vanishing point direction increases.

  After calculating the lost energy VE in the current processing target frame image in s11, the image processing unit 4 performs time series smoothing processing of the lost energy VE (s12). The time series smoothing process for s12 is smoothed by using the lost energy VE calculated in a predetermined sampling period (past period) based on the lost energy VE calculated this time by a known least square method or the like. It is a process to convert.

  The image processing unit 4 presets an approach threshold, a departure threshold, and a stop threshold (entrance threshold> departure threshold> stop threshold) for the value of the lost energy VE smoothed in s12. Further, the image processing unit 4 presets a presence threshold and a passage threshold (existence threshold> passing threshold) with respect to the loss energy VE calculated in s11 (the loss energy VE not smoothed in s12).

  If it is determined that the train is not captured (no train) with respect to the previous processing target frame image, the image processing unit 4 determines whether the train has entered (s13, s14). In s14, if the value of the lost energy VE subjected to the time series smoothing process in s12 is equal to or greater than a preset entry threshold value, the current state is determined to be a train entry. If this is the case, it is determined that there is no train. The image processing unit 4 stores the current determination result in s14 in the memory (s15), and returns to s11.

  Further, when the image processing unit 4 determines that the train has entered the previous processing target frame image, the image processing unit 4 performs the train passage determination (s13, s16). In s16, the disappearance energy VE calculated in s11 is always equal to or greater than the preset existence threshold during the existence time determined with reference to the imaging timing of the current processing target frame image, and the disappearance calculated in s11 this time. If the energy VE is less than a preset passage threshold, it is determined that the train has passed. The existence time is a time period from t-T1 to t-T2, where t is the imaging timing of the current processing target frame image. T1 and T2 are predetermined times, and T1> T2. If the image processing unit 4 determines that the train has passed in the train passage determination for s16, the image processing unit 4 stores the current determination result (no train) in the memory (s17, s18), and returns to s11.

  On the other hand, the image processing unit 4 performs the train stop determination (s17, s19) unless it is determined that the train has passed in the train passage determination in s16. In s19, if the value of the lost energy VE subjected to the time series smoothing process in s12 this time is less than a predetermined stop threshold, the current state is determined to be a train stop, and the current process target frame image is used as a reference. It is stored as a frame image (s20, s22). On the other hand, if the value of the lost energy VE subjected to the time series smoothing process at s12 this time is not less than the predetermined stop threshold value, the image processing unit 4 determines that the current state is approaching the train, and this determination The result is stored in the memory (s20, s21), and the process returns to s11.

  After storing the reference frame image in s22, the image processing unit 4 ends the stop detection process for s1, and proceeds to the state determination process for s2. That is, the image processing unit 4 repeats the stop detection process (the process shown in FIG. 4) for s1 until the stop of the train at the station platform is detected.

  In this example, the image processing unit 4 processes the frame image input to the image input unit 3 and detects the stop of the train at the station platform. However, is the train stopped at the station platform? It is good also as a structure which inputs the stop notification signal which shows whether to the input-output part 5 from a high-order apparatus. When configured in this way, the image processing apparatus 1 determines the timing at which the stop notification signal input from the host device changes to a signal indicating that the train has stopped at the station platform as the stop timing, What is necessary is just to determine the timing which changed to the signal which shows that the train has not stopped at the home as the departure timing. Further, the image processing apparatus 1 may use the frame image whose imaging timing is immediately after the stop timing as the reference frame image stored in s22 described above. Further, since it is not necessary to perform the processing shown in FIG. 4 in the image processing unit 4, the imaging device 10 may not be attached to an angle at which a vanishing point of a train that is a moving body is generated. That is, the angle of the imaging device 10 may be perpendicular to the moving direction (traveling direction) of the train.

  Further, the train stop detection in the image processing apparatus 1 may be a method other than the above.

  Next, the state determination process concerning s2 will be described with reference to FIG. FIG. 5 is a flowchart showing the state determination process.

  First, the reference frame image stored in s22 by the image processing unit 4 will be described. The train opens the train vehicle door after stopping at the station platform. Therefore, in s22, it is the frame image which imaged the state before the door of the train vehicle stopped at the station (that is, the state where the train vehicle door is closed). That is, the reference frame image is a frame image in a state where the train is stopped at the station platform and the door closing indicator light is turned off. The processing according to s22 is a configuration corresponding to the reference frame image acquisition unit referred to in the present invention.

  The image processing unit 4 waits for a determination start timing to start determining whether the door closing indicator light is on or off (s31). The image processing unit 4 determines the determination start timing based on the imaging timing of the reference frame image stored in s22. The determination start timing is, for example, a timing after a fixed time has elapsed from the reference frame image capturing timing. The certain time may be defined by the number of frames of the frame image input to the image input unit 3.

  s31 is a process of waiting for the door of the train that has detected that the vehicle has stopped at the station platform to open in the stop detection process according to s1, and for the door closing indicator light attached to the side surface of the train to light up. . That is, s31 is a step provided to start the processing after s32 shown below after the door closing indicator light of the vehicle of the train stopped at the station platform is turned on.

  The waiting time in s31 is set according to the station platform where the train stops. In a general station, since the time from when the train stops at the station platform until the door passes is several seconds, the waiting time in s31 may be set to several seconds (about 1 to 5 seconds). On the other hand, at a station where the passenger's disembarking platform differs from the passenger's platform (for example, at the end / starting station of the route on which the train is looped back), when the train stops at the station platform, Open the door on the side, get off the passengers in the train, and then open the door on the boarding platform side. Therefore, at the station where the passenger's getting-off platform and the passenger's boarding platform are different, the waiting time in s31 is several minutes (1 About 5 minutes).

  At the determination start timing, the image processing unit 4 extracts a processing target frame image to be processed this time from the frame images input to the image input unit 3, and determines whether or not the train that has detected a stop has left the station platform. Determine (s32). Also in this state determination process, the image processing unit 4 selects processing target frame images in time series as in the above-described stop detection process. The image processing unit 4 sequentially extracts the frame images input to the image input unit 3 for every m frames as processing target frame images. m is an integer of 1 or more.

  In s32, the same process as the process concerning s11 and s12 described above is performed, and if the value of the lost energy VE subjected to the time series smoothing process is equal to or greater than a predetermined departure threshold, it is determined that the train has started.

  Note that the image processing unit 4 performs the determination related to s32 while waiting for the determination start timing in s31 described above.

  When determining that the train has not started in s32, the image processing unit 4 performs vehicle-side lamp detection preprocessing for the current processing target frame image (s33). FIG. 6 is a flowchart showing the vehicle side light detection pre-processing according to s33.

  The image processing unit 4 converts the current processing target frame image into a grayscale image (hereinafter also referred to as a processing target reference grayscale image) (s51). As described above, the imaging device 10 is a color video camera, and the frame image input to the image input unit 3 is a color image.

In s51, the luminance value Y (i, j) of each pixel on the processing target reference grayscale image is calculated.
Y (i, j) = a * B (i, j) + b * G (i, j) + c * R (i, j) + d
Calculated by
(I, j) indicates the position of the pixel on the frame image.
B (i, j) is a blue pixel value on the processing target frame image, G (i, j) is a green pixel value on the processing target frame image, and R (i, j) is a processing target frame image. The upper red pixel value is shown.

  The coefficients a, b, c, and d are predetermined values. In this example, since the lighting color of the door closing indicator for determining the lighting state is red, the influence of the red pixel value on the luminance value Y is greater than the blue pixel value or the green pixel value. Thus, coefficients a, b, c, and d are determined. Specifically, the value of the coefficient c is set to a value larger than the coefficients a and b. For example, a = 0.098, b = 0.257, c = 0.504, d = 16.

  If the frame image input to the image input unit 3 is not a color image but a black and white image, the process according to s51 is not necessary.

  The image processing unit 4 also applies a grayscale image of the reference frame image (hereinafter also referred to as a reference grayscale image) obtained by performing the processing according to s51 on the reference frame image stored in s22 described above. I remember it. This process is performed using the time waiting for the determination start timing in s31.

  The processing target grayscale image and the reference grayscale image are, for example, images of 256 gradations.

  The image processing unit 4 generates a luminance difference image between the reference grayscale image and the processing target grayscale image, and extracts pixels having a luminance difference larger than a predetermined threshold (s52). The image processing unit 4 performs noise removal processing for removing noise pixels from the pixels extracted in s52 (s53). The noise removal processing according to s53 is processing for removing, as noise pixels, pixels that are isolated on the processing target frame image from the pixels extracted at s52.

  The image processing unit 4 performs a mask process for masking a preset image area on the luminance difference image subjected to the noise removal process in s53 (s54). In s54, the region where the door closing indicator lamp is not imaged is excluded from the processing target range on the frame image. That is, the process concerning s54 is a process which removes a pixel which is clearly not a door closing indicator lamp from a position on the frame image even if it is a pixel extracted in s52 and not removed as noise in s53. For example, in s54, a region surrounded by a broken line in FIG. 7A (a region where the candidate regions A to C shown in FIG. 7A are located and the door closing indicator may be imaged) Mask processing is performed to remove other than the range to be processed. Thereby, the candidate area | region D shown to FIG. 7 (A) can be processed as a noise.

  Note that the candidate areas A to D are areas where the luminance difference image generated in s52 is accumulated to the extent that there are pixels whose luminance difference is larger than a predetermined threshold. That is, all the pixels located in the candidate areas A to D are not pixels whose luminance difference is larger than a predetermined threshold in the luminance difference image generated in s52.

  As shown in FIG. 7, the candidate areas A and B are areas where the door closing indicator lamp is imaged, but the candidate area C is a noise area where the door closing indicator lamp is not captured (disturbance light is irradiated, The area where the brightness has changed.

  The image processing unit 4 generates a binarized image subjected to mask processing (s55). The binarized image generated in s55 is an image in which, for example, the pixels having a luminance difference in the candidate areas A to C illustrated in FIG.

  The image processing unit 4 performs a labeling process for detecting the pixel block extracted in s52 as one object in the binarized image generated in s55 (s56).

  That is, in the vehicle side light detection pre-processing for s33, a candidate area (an area where the object labeled in s56 is imaged) that is likely to be imaged is displayed. .

  The process illustrated in FIG. 6 is an example, and the luminance difference image generated in s52 may be a binarized image. In this case, the processing relating to s55 can be made unnecessary. In addition, the mask processing related to s54 may be performed before the noise removal processing is performed in s53. Moreover, as long as the candidate area in which the lit door closing indicator lamp is imaged can be detected for the processing target frame image, the processing may be performed by another process instead of the process illustrated in FIG. .

  The image processing unit 4 performs the first state determination process for each candidate area detected in s33 (s34). In the first state determination process, it is determined whether the candidate area detected in s33 is an area in which a lit door closing indicator is imaged, and the lit door closing indicator is imaged. If it is an area, the process registers the area as a lighting area. FIG. 8 is a diagram illustrating the first state determination process. FIG. 8 shows processing for one candidate area detected in s33. In s34, the process shown in FIG. 8 is executed for each candidate area detected in s33.

  The image processing unit 4 determines whether the candidate area to be processed is a lighting area that has already been registered in the state determination process for the previous process target frame image (s61). If there is an overlapping area between the candidate area to be processed and the already registered lighting area, the image processing unit 4 determines that the lighting area has already been registered. That is, in s61, it is determined whether or not the lighting region is already registered depending on whether or not there is a lighting region already registered in the vicinity of the candidate region to be processed.

  If the image processing unit 4 determines in s61 that the candidate region to be processed is not a lighting region that is registered, it determines whether or not a new registration condition is satisfied (s62).

  In s62, a resizing process is performed to match the size of the candidate area with the size of the door closing indicator light being imaged. In the time zone such as evening or night when the periphery of the door closing indicator is relatively dark, the candidate area detected by the labeling process in s56 may be larger than the area where the door closing indicator is imaged. (It may be an area including the periphery of the door closing indicator.) This is because light is diffused when the door closing indicator lamp is turned on. The resizing process is a process of cutting an area where the luminance is somewhat low in the candidate area, and is a process of cutting a portion where light is diffused in the candidate area.

  The resizing process will be described by taking as an example a case where a candidate area indicated by hatching in FIG. 9A is detected in s33 in the current processing target frame image. As shown in FIG. 9A, the image processing unit 4 sets a rectangular area inscribed by the candidate area as a conversion area. FIG. 9B is an enlarged view showing the conversion area.

  The set conversion area has a size of n pixels in the horizontal direction and m pixels in the vertical direction (n × m pixels in size). The image processing unit 4 calculates a vertical luminance difference for each column in which pixels are arranged in the vertical direction in the conversion area. Specifically, for each column in which pixels are arranged in the vertical direction in the conversion region, the image processing unit 4 determines the luminance of the corresponding pixel in the reference frame image stored in s22 described above for each pixel positioned in that column. Calculate the difference. The vertical luminance difference is the sum of the luminance differences calculated for each pixel located in the column for each column in which pixels are arranged in the vertical direction in the conversion region.

  For each column in which pixels are arranged in the vertical direction in the conversion area, the image processing unit 4 calculates the vertical luminance difference calculated for that column by the number of pixels located in that column (m in the example shown in FIG. 9B). The average luminance difference in the vertical direction divided by The image processing unit 4 calculates half (1/2) of the difference between the maximum value and the minimum value of the average luminance difference in the vertical direction of each column arranged in the vertical direction of the conversion area as the vertical direction division threshold value.

Further, the image processing unit 4 calculates, for each column arranged in the vertical direction of the conversion area, the vertical luminance difference average of the column,
Average vertical luminance difference> (Minimum average vertical luminance difference) + (Vertical division threshold)
Extract columns that satisfy. The image processing unit 4 determines a resize width between the rightmost column satisfying the above condition and the leftmost column in the conversion area.

  Further, the image processing unit 4 calculates a horizontal luminance difference for each row in which pixels are arranged in the horizontal direction in the conversion region. Specifically, for each row in which pixels are arranged in the horizontal direction in the conversion region, the image processing unit 4 determines the luminance of the corresponding pixel in the reference frame image stored in s22 described above for each pixel located in that row. Calculate the difference. The horizontal luminance difference is the sum of the luminance differences calculated for each pixel located in a row for each row of pixels arranged in the horizontal direction in the conversion region.

  For each row in which pixels are arranged in the horizontal direction in the conversion area, the image processing unit 4 calculates the horizontal luminance difference calculated for that row by the number of pixels located in that row (n in the example shown in FIG. 9B). The average luminance difference in the horizontal direction divided by The image processing unit 4 calculates a half (1/2) of the difference between the maximum value and the minimum value of the horizontal luminance difference average of each row arranged in the horizontal direction of the conversion area as the horizontal division threshold value.

Further, the image processing unit 4 calculates, for each column arranged in the horizontal direction of the conversion area, the horizontal luminance difference average of the column,
Horizontal luminance difference average> (Minimum horizontal luminance difference average) + (Horizontal division threshold)
Extract rows that satisfy. The image processing unit 4 determines the resize height between the uppermost row satisfying the above condition and the lowermost row in the conversion area.

  Then, the image processing unit 4 sets the rectangular area surrounded by the resize width and the resize height determined in the above process as a resize area.

The image processing unit 4 determines whether the determined resize area satisfies all of the following three new registration conditions (new registration condition 1 to new registration condition 3).
New registration condition 1: minimum value of width ≦ width of resized area ≦ maximum value of width,
And the minimum value of height ≦ the height of the resized area ≦ the maximum value of height.
New registration condition 2: A pixel in which more than half of the pixels in the resize area are extracted in s52 (a pixel having a luminance difference larger than a predetermined threshold).
New registration condition 3: The average luminance value of the pixels in the region corresponding to the resized region in the reference frame image is equal to or less than a predetermined average luminance value.

The new registration condition 1 is to determine whether or not the lit door closing indicator lamp is imaged based on the size of the resize area. Whether the new registration condition 2 is a change in luminance value due to lighting of the door closing indicator light or a change in luminance value due to another factor (other than lighting of the door closing indicator light) such as irradiation of ambient light is determined. Is. Moreover, the new registration condition 2, but more than half of the pixels in the resize area is assumed to be the pixel extracted in s52, the ratio of the pixel or 2/3 or more, may be 1/3 or more and the like. The new registration condition 3 is for determining whether or not the candidate area is a relatively bright area by irradiation with disturbance light or the like.

  If the image processing unit 4 determines that all of the new registration conditions 1 to 3 are satisfied, the image processing unit 4 registers the resize area related to the candidate area that is the current processing target as the lighting area (s63), and ends this process. To do. The image processing unit 4 stores the position on the frame image of the resized area registered as the lighting area in s63. On the other hand, if the image processing unit 4 determines in s62 that the new registration condition is not satisfied, the image processing unit 4 ends this process without registering the candidate area that is the current process target as the lighting area.

  Note that here, when all of the new registration conditions 1 to 3 are satisfied, the resizing area for the candidate area that is the current processing target is registered as the lighting area. It may be configured to register as a lighting area when two new registration conditions are satisfied, or may be registered as a lighting area when two or more new registration conditions are satisfied.

  If the image processing unit 4 determines in s61 that the candidate region to be processed is a lighting region that has already been registered, the image processing unit 4 determines whether or not the continuation condition is satisfied (s64). In s64, the resizing process described above may or may not be performed on the candidate area. Here, description will be made assuming that the resizing process is performed.

The image processing unit 4 determines whether or not both of the following two continuation conditions (continuation condition 1 and continuation condition 2) are satisfied.
Continuation condition 1: At the time of registration of the lighting region, the number of pixels (Y) located in the current resize region relative to the number of pixels (X) extracted in s52 that are located in the lighting region The ratio (Y / X) is a certain value (for example, 0.8) or more.
Continuation condition 2: There is no region whose luminance value is similar to the current resized region near the registered region on the reference frame image.

  In continuation condition 2, the search range on the reference frame image for searching for a similar region is set to the size shown in FIG. The search range is a rectangular area whose center coincides with the center of the lighting area, has a width twice that of the lighting area, and a height three times that of the lighting area. The similarity is calculated using a known zero average normalized cross-correlation or the like. The height of the search range is increased in consideration of the shaking of the train as passengers get on and off. This search range is not limited to the above size.

  Although the description has been given here assuming that the resizing process is performed, if the resizing process is not performed, the resizing area according to the continuation condition may be replaced with a candidate area.

  If at least one of the two continuation conditions (continuation condition 1 and continuation condition 2) is not satisfied in s64, the image processing unit 4 deletes the registration of the corresponding lighting region (s66), and ends this process. That is, at the time of registration of the lighting area, the image processing unit 4 counts the number of pixels located in the current resize area relative to the number of pixels (X) extracted in s52 that are located in the lighting area. When the ratio (Y / X) of (Y) is not a certain value (for example, 0.8) or more (when the above continuation condition 1 is not satisfied), or near the registered area on the reference frame image, the luminance value is If there is a region similar to the resize region (when the above continuation condition 2 is not satisfied), the corresponding lighting region registration is deleted in s66.

  If the image processing unit 4 determines that both of the two continuation conditions (continuation condition 1 and continuation condition 2) are satisfied in s64, the image processing unit 4 increments the number of continuation frames for the corresponding lighting region by 1 (s65), and ends this process. To do.

  Here, the image processing unit 4 is configured to be determined to continue when both the continuation condition 1 and the continuation condition 2 are satisfied, but is determined to continue when either one is satisfied. It is good also as a structure.

  When completing the first state determination process in s34, the image processing unit 4 performs the second state determination process. The second state determination process is a process of determining whether or not the door closing indicator lamp that has been in the lighting state in the previous processing target frame image has been turned off. This process is performed for each lighting region registered at this time. FIG. 11 is a flowchart showing the second state determination process. FIG. 11 shows processing for one lighting area registered at this time. That is, in s35, the process shown in FIG. 11 is performed for every lighting area registered at this time.

  The image processing unit 4 determines whether or not the lighting area to be processed is determined to continue in s34 (s71). If it is determined that the lighting area to be processed continues in s34, the image processing unit 4 ends this process.

  On the other hand, if it is not determined that the lighting area to be processed continues in s34, the image processing unit 4 performs a light-off determination to determine whether the light-off condition is satisfied (s72). In the turn-off determination according to s72, the resizing process described above may or may not be performed. Here, description will be made assuming that the resizing process is performed.

The image processing unit 4 determines whether all of the following three extinguishing conditions (extinguishing condition 1 to extinguishing condition 3) are satisfied.
Extinguishing condition 1: The lighting area to be processed is compared with the lighting area on the current processing target frame image, and the number of pixels whose luminance values are reduced in the current processing target frame image is a predetermined number. Has reached.
Light-off condition 2: Near the lighting area on the current processing target frame image, there is an area whose luminance value is similar to the lighting area on the reference frame image.
Light-off condition 3: The lighting area is registered over a predetermined number of frames.

  As with the continuation condition 2 described above, the determination regarding the turn-off condition 2 is performed by setting a search range for searching for a similar region on the processing target frame image and using a known zero average normalized cross-correlation or the like. Just do it.

  If the image processing unit 4 determines in s72 that all three extinguishing conditions (extinguishing condition 1 to extinguishing condition 3) are satisfied, the image processing unit 4 determines that the door closing indicator lamp in the lighting area to be processed is extinguished. . On the other hand, if the image processing unit 4 determines in s72 that any one of at least three extinguishing conditions (extinguishing condition 1 to extinguishing condition 3) is not satisfied, the lighting area to be processed is registered by erroneous detection. It is determined that it is the lighting area of the door closing indicator lamp.

  When the image processing unit 4 completes the turn-off determination in s72, it deletes the registration of the lighting area to be processed (s73), and ends this process.

  Here, when all of the above-described extinguishing condition 1 to extinguishing condition 3 are satisfied, it is determined that the door closing indicator lamp is extinguished with respect to the lighting area to be processed this time. When the light-off condition is satisfied or when two or more light-off conditions are satisfied, it may be determined that the door closing indicator light is turned off with respect to the lighting region that is the current processing target.

  When the image processing unit 4 finishes the second state determination process for s35, the image processing apparatus 1 determines whether there is a door closing indicator that has been determined to be turned off this time (s36). If there is a door closing indicator that is determined to be turned off this time, the image processing apparatus 1 outputs a signal indicating the door closing from the input / output unit 5 to the host device (s37), returns to s32, and repeats the above-described processing. . On the other hand, if there is no door closing indicator that is determined to be turned off this time, the image processing apparatus 1 returns to s32 without outputting a signal indicating that the door is closed from the input / output unit 5 to the host device in s37, and performs the above-described processing. repeat.

  When the image processing unit 1 determines that the train has started at s32, the image processing device 1 ends this processing. Thereby, the image processing apparatus 1 starts the stop detection process concerning s1.

  Thus, this image processing apparatus 1 is a frame image in which the train vehicle door is closed, that is, a frame image in which the door closing indicator for determining the state (lighted state or unlit state) is turned off, It is acquired as a reference frame image, and it is determined from the brightness difference image with this reference frame image whether the door closing indicator lamp is in the lit state or the extinguished state. Thereby, it can prevent misdetecting that the location where the disturbance light (sunlight or the illumination light of a station platform) is irradiated is the location where the door closing indicator light is lit. Therefore, the state of the door closing indicator lamp can be accurately determined.

  In addition, as described above, the image processing apparatus 1 according to this example sends a signal indicating that the door is closed from the input / output unit 5 when any of the door closing indicator lights shifts from the lighting state to the extinguishing state. Output to. Therefore, when the image processing apparatus 1 according to this example outputs a signal indicating that the door is closed from the input / output unit 5 to the host device, not all the door closing indicator lights imaged by the imaging device 10 are turned off. Absent. Normally, a train closes a door of a vehicle that is open in preparation for departure at substantially the same timing. However, a passenger's bag etc. may be pinched | interposed into a door and a door may not close with some vehicles.

  That is, the signal indicating that the door is closed, which is output from the input / output unit 5 to the host device by the image processing apparatus 1 according to this example, notifies that the train is about to start, and indicates whether there is a door that is not closed. Not a notification. Therefore, the host device does not permit the departure of the train by the input of the signal indicating that the door is closed from the image processing device 1. The host device prepares an input of a signal indicating that the door is closed from the image processing device 1, for example, at the departure of a train, and alerts a passenger who stays at the station platform (for example, “ , The train departs from the station. Please go down to the white line. ”This message is used to determine when to start broadcasting and when to close the home door.

  The image processing apparatus 1 may replace the state determination process illustrated in FIG. 5 with the state determination process illustrated in FIG. Hereinafter, the state determination process illustrated in FIG. 12 will be described.

  The image processing apparatus 1 according to this example also performs the processes of s31 to s35 illustrated in FIG. When completing the second state determination process in s35, the image processing apparatus 1 determines the presence / absence of a door-closed indicator lamp that is in a lit state (s136). If the image processing apparatus 1 determines that there is no door-closed indicator lamp that is lit, the image processing apparatus 1 outputs a signal indicating the door closing from the input / output unit 5 to the host device (s137), returns to s32, and repeats the above-described processing. . On the other hand, if the image processing apparatus 1 determines that there is a door-closed indicator lamp that is lit, the image processing apparatus 1 returns to s32 without outputting a signal indicating that the door is closed from the input / output unit 5 to the host apparatus in s137, and the above-described processing repeat.

  The image processing apparatus 1 according to this example outputs a signal indicating that the door is closed from the input / output unit 5 to the host device when all the door closing indicator lights imaged by the imaging device 10 are turned off. Therefore, the host device can determine whether to permit departure of the train (whether all the vehicle doors of the train are closed) using the signal indicating the door closing input from the image processing device 1. .

  The invention of the present application is not limited to the apparatus for judging the state of the door closing indicator lamp attached to the side surface of the train vehicle shown in the above example, but also the emergency light attached to the side surface of the train vehicle. It can also be used as a device for determining the state.

  Moreover, it can utilize also as an apparatus which determines lighting states, such as an external light and a signal lamp device.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 2 ... Control part 3 ... Image input part 4 ... Image processing part 5 ... Input / output part 10 ... Imaging device

Claims (18)

A lighting state determination device that determines whether a side lamp attached to a side surface of a moving body is in a lighting state or a lighting state,
A frame image input unit for inputting a frame image obtained by imaging the side surface of the moving body;
Based on a frame image input to the frame image input unit, in which the moving body being imaged is in a stopped state and a side lamp attached to a side surface of the moving body is in an off state. A reference frame image acquisition unit to acquire a frame image;
A processing target frame image acquisition unit that acquires a frame image that is input to the frame image input unit and that is captured after the reference frame image acquired by the reference frame image acquisition unit as a processing target frame image; ,
A luminance difference image generation unit that generates a luminance difference image between the reference frame image and the processing target frame image;
In the luminance difference image generated by the luminance difference image generation unit, an area where a predetermined luminance difference is generated is extracted as a candidate area, and whether or not a side lamp that is lit is captured in the extracted candidate area and a state determination section determines,
The state determination unit has a resizing function for resizing the size of the extracted candidate area according to the diffusion of light in the candidate area,
The lighting state determination apparatus which determines whether the side lamp which is in a lighting state is imaged in the resizing area which resized the candidate area by the resizing function . When the size of the resized area obtained by resizing the candidate area by the resize function is not within a predetermined size range, the state determination unit determines that a side lamp that is in a lighting state is not captured in the resized area. The lighting state determination apparatus according to claim 1 , wherein the determination is performed. The state determination unit determines in advance a ratio of pixels in which a predetermined luminance difference is generated in the luminance difference image generated by the luminance difference image generation unit in the resized region obtained by resizing the candidate region by the resize function. The lighting state determination device according to claim 1 or 2 , wherein if the ratio is not equal to or greater than a certain ratio, it is determined that a side lamp that is in a lighting state is not imaged in the resized area. If the average luminance value of the area on the reference frame image to which the resized area obtained by resizing the candidate area by the resize function exceeds a predetermined luminance value, the state determination unit is in a lighting state in the resized area The lighting state determination apparatus according to any one of claims 1 to 3 , wherein it is determined that a side lamp that is is not captured. A registration unit for registering, as a lighting region, a region where the state determination unit determines that a side lamp attached to a side surface of the moving body is in a lighting state;
The state determination unit determines, for each lighting region registered by the registration unit, whether or not a side lamp located in the lighting region is extinguished,
The lighting state determination device according to any one of claims 1 to 4 , wherein the registration unit deletes registration of an area in which the state determination unit determines that the side lamp is extinguished. A lighting state determination device that determines whether a side lamp attached to a side surface of a moving body is in a lighting state or a lighting state,
A frame image input unit for inputting a frame image obtained by imaging the side surface of the moving body;
Based on a frame image input to the frame image input unit, in which the moving body being imaged is in a stopped state and a side lamp attached to a side surface of the moving body is in an off state. A reference frame image acquisition unit to acquire a frame image;
A processing target frame image acquisition unit that acquires a frame image that is input to the frame image input unit and that is captured after the reference frame image acquired by the reference frame image acquisition unit as a processing target frame image; ,
A luminance difference image generation unit that generates a luminance difference image between the reference frame image and the processing target frame image;
In the luminance difference image generated by the luminance difference image generation unit, an area where a predetermined luminance difference is generated is extracted as a candidate area, and whether or not a side lamp that is lit is captured in the extracted candidate area A state determination unit for determining
A registration unit that registers, as a lighting region, a region in which the state determination unit determines that a side lamp attached to a side surface of the moving body is in a lighting state;
The state determination unit determines, for each lighting region registered by the registration unit, whether or not a side lamp located in the lighting region is extinguished,
The said registration part is a lighting state determination apparatus which deletes registration of the area | region determined in the said state determination part as the side lamp having extinguished . The state determination unit, before SL case of extracting a candidate region which overlaps with the lighting area registered by the registering unit, the lighting on the luminance difference image the luminance difference image generation unit when registering the lighting region is generated The luminance difference image generated this time by the luminance difference image generation unit with respect to the number of pixels that are located within the region and have a predetermined luminance difference with the corresponding pixel on the reference frame image If the ratio of the number of pixels that are located in the upper candidate region and have a predetermined luminance difference with the corresponding pixel on the reference frame image is not greater than a certain value, The lighting state determination apparatus according to claim 5, wherein the registration unit deletes registration of the lighting region . The state determination unit, when extracting the pre Symbol candidate region overlaps the lighting area registered by the registering unit, the lighting near the area on the reference frame image, similar to the candidate region in which the luminance value is extracted The lighting state determination apparatus according to claim 5 , wherein if there is a region, the registration unit deletes the registration of the lighting region . When the state determination unit does not extract a candidate region that overlaps the lighting region registered by the registration unit, the lighting region on the previous processing target frame image and the lighting on the current processing target frame image If the number of pixels whose luminance value has decreased in the current processing target frame image does not reach a predetermined fixed number, the side lamp located in the lighting area is not turned off. The lighting state determination apparatus according to any one of claims 5 to 8 , wherein the determination is performed. When the state determination unit does not extract a candidate region that overlaps the lighting region registered by the registration unit, the luminance value is near the lighting region on the current processing target frame image, and the luminance value is on the reference frame image. The lighting state determination device according to claim 5 , wherein if there is no region similar to the lighting region, it is determined that the side lamp located in the lighting region is not turned off. If the state determination unit does not extract a candidate region that overlaps the lighting region registered by the registration unit, the lighting region is not registered as a side lamp is lit for a predetermined number of frames. The lighting state determination device according to any one of claims 5 to 10 , wherein it is determined that a side lamp located in the lighting region is not turned off. In the frame image input unit, an input frame image is a color image,
The luminance difference image generation unit converts the reference frame image acquired by the reference frame image acquisition unit into a grayscale reference grayscale image, and converts the processing target frame image acquired by the processing target frame image acquisition unit to grayscale. Has a grayscale conversion function to convert to a target grayscale image
The lighting according to claim 1, wherein a difference image between the reference grayscale image converted to grayscale by the grayscale conversion function and the processing target grayscale image is generated as a luminance difference image. State determination device. In the conversion from a color image to a grayscale image, the grayscale conversion function has an influence on the luminance value of the converted grayscale image by the lighting color of the side lamp attached to the side surface of the moving body. The lighting state determination device according to claim 12 , wherein the lighting state determination device has a larger effect than the luminance value of the grayscale image into which the color has been converted. From a moving state in which the moving body is moving, comprising a stop detection unit that detects that the moving body has stopped.
The reference frame image acquisition unit is a frame image input to the frame image input unit, and is captured at a timing when the stop detection unit detects that the moving body has changed from a moving state to a stopped state. acquires frame image as the reference frame image, the lighting state determination device according to any one of claims 1 to 13. A frame image obtained by imaging the side surface of the moving object input to the frame image input unit, and determining whether the side lamp attached to the side surface of the moving object is in the lit state or in the unlit state A method,
Based on a frame image input to the frame image input unit, in which the moving body being imaged is in a stopped state and a side lamp attached to a side surface of the moving body is in an off state. A reference frame image obtaining step for obtaining a frame image;
A frame image input to the frame image input unit, a processing target frame image obtaining step of obtaining a frame image captured in the reference frame image later acquired by the reference frame image obtaining step as the processing target frame image ,
A luminance difference image generating step for generating a luminance difference image between the reference frame image and the processing target frame image;
In the luminance difference image generated in the luminance difference image generation step, an area where a predetermined luminance difference is generated is extracted as a candidate area, and whether or not a side lamp that is lit is captured in the extracted candidate area and a state determination step of determining,
The state determination step includes a process of resizing the size of the extracted candidate area according to the diffusion of light in the candidate area,
A lighting state determination method , which is a step of determining whether or not a side lamp that is in a lighting state is imaged in a resized area obtained by resizing the candidate area by the resizing process. A frame image obtained by imaging the side surface of the moving object input to the frame image input unit, and determining whether the side lamp attached to the side surface of the moving object is in the lit state or in the unlit state A method,
Based on a frame image input to the frame image input unit, in which the moving body being imaged is in a stopped state and a side lamp attached to a side surface of the moving body is in an off state. A reference frame image obtaining step for obtaining a frame image;
A processing target frame image acquisition step of acquiring a frame image input to the frame image input unit and captured as a processing target frame image after the reference frame image acquired in the reference frame image acquisition step; ,
A luminance difference image generating step for generating a luminance difference image between the reference frame image and the processing target frame image;
In the luminance difference image generated in the luminance difference image generation step, an area where a predetermined luminance difference is generated is extracted as a candidate area, and whether or not a side lamp that is lit is captured in the extracted candidate area A state determination step for determining
A registration step of registering in the registration unit as a lighting region an area where it is determined that the side lamp attached to the side surface of the moving body is in a lighting state in the state determination step;
The state determination step is a step of determining whether or not a side lamp located in the lighting region is extinguished for each lighting region registered in the registration unit,
The said registration step is a lighting state determination method which is a step including the process which deletes the registration of the area | region determined that the side lamp turned off in the said state determination step. Processing a frame image obtained by capturing the side surface of the moving body input to the frame image input unit, and determining whether the side lamp attached to the side surface of the moving body is in a lit state or in an unlit state A lighting state determination program to be executed by
Based on a frame image input to the frame image input unit, in which the moving body being imaged is in a stopped state and a side lamp attached to a side surface of the moving body is in an off state. A reference frame image obtaining step for obtaining a frame image;
A processing target frame image acquisition step for acquiring a frame image input to the frame image input unit and captured as a processing target frame image after the reference frame image acquired by the reference frame image acquisition step; ,
A luminance difference image generating step for generating a luminance difference image between the reference frame image and the processing target frame image;
In the luminance difference image generated in the luminance difference image generation step, an area where a predetermined luminance difference is generated is extracted as a candidate area, and whether or not a side lamp that is lit is captured in the extracted candidate area a state determination step of determining, cause the computer to execute,
The state determination step includes a process of resizing the size of the extracted candidate area according to the diffusion of light in the candidate area,
The lighting state determination program which is a step which determines whether the side lamp which is in a lighting state is imaged in the resizing area which resized the candidate area by the resizing process . Processing a frame image obtained by capturing the side surface of the moving body input to the frame image input unit, and determining whether the side lamp attached to the side surface of the moving body is in a lit state or in an unlit state A lighting state determination program to be executed by
Based on a frame image input to the frame image input unit, in which the moving body being imaged is in a stopped state and a side lamp attached to a side surface of the moving body is in an off state. A reference frame image obtaining step for obtaining a frame image;
A processing target frame image acquisition step for acquiring a frame image input to the frame image input unit, the frame image captured after the reference frame image acquired by the reference frame image acquisition step as a processing target frame image; ,
A luminance difference image generating step for generating a luminance difference image between the reference frame image and the processing target frame image;
In the luminance difference image generated in the luminance difference image generation step, an area where a predetermined luminance difference is generated is extracted as a candidate area, and whether or not a side lamp that is lit is captured in the extracted candidate area A state determination step for determining
A registration step of registering in the registration unit as a lighting region an area determined that the side lamp attached to the side surface of the moving body is in a lighting state in the state determination step is executed by a computer,
The state determination step is a step of determining whether or not a side lamp located in the lighting region is extinguished for each lighting region registered in the registration unit,
The said registration step is a lighting state determination program which is a step including the process which deletes the registration of the area | region determined that the side lamp turned off in the said state determination step.

Images