JP7567294B2 - Judgement support system and judgement support method - Google Patents

Description

This disclosure relates to a decision support system and a decision support method.

It is known that in ball games, track and field, martial arts and other sports, when judging a specific movement near a boundary line, such as whether or not the ball or the player's body has gone outside the boundary line, recorded video of a specific area including the boundary line is used to assist the judge. For example, Patent Document 1 discloses a judgment assistance system that extracts and presents scenes that are useful for the judge's judgment from video data taken by multiple cameras when making an in-out judgment on whether a ball (including a feather ball) has landed inside or outside the court in ball games such as volleyball.

JP 2018-137671 A

When using a judgment assistance system like the one described above, a referee or other person makes a judgment by grasping the positional relationship between the boundary line and the judgment object, such as a sphere or part of a body, from the played-back recorded video. However, if the movement speed of the judgment object in the recorded video is fast, it can be difficult and time-consuming to play back the part of the recorded video that is effective for judgment and to judge the positional relationship of the judgment object in relation to the boundary line.

The present disclosure has been made in consideration of the above, and aims to provide a judgment assistance system and judgment assistance method that can easily determine the positional relationship between an object to be judged and a specified boundary line during a competition.

To achieve the above object, a judgment assistance system according to one embodiment of the present disclosure includes a projection image generation unit that generates projection information for projecting a projection image along a boundary line that divides a competition area used in a competition, an object detection unit that detects an image area of an object to be judged from a captured image that includes the boundary line, a projection judgment unit that judges whether the projection image is detected in the image area of the object to be judged, and a detection information generation unit that generates detection information when the projection image is detected in the image area of the object to be judged.

To achieve the above object, a judgment assistance method according to one aspect of the present disclosure includes a projection image generation step of generating projection information for projecting a projection image along a boundary line that divides a competition area used in a competition, an object detection step of detecting an image area of an object to be judged from a captured image that includes the boundary line, a projection step of determining whether or not the projection image is detected in the image area of the object to be judged, and a detection information generation step of generating detection information when the projection image is detected in the image area of the object to be judged.

The present disclosure has the effect of shortening the time required to determine the positional relationship between an object to be determined and a specified boundary line during a competition.

FIG. 1 is a plan view illustrating a decision support system according to an embodiment. FIG. 2 is a diagram showing an example of a projected image. FIG. 3 is a diagram showing an example of a projected image. FIG. 4 is a functional block diagram illustrating an example of the configuration of a judgment assistance system according to an embodiment. FIG. 5 is a diagram showing an example of a captured image of a sphere. FIG. 6 is a diagram showing an example of the additional information. FIG. 7 is a flowchart illustrating an example of a process performed by the decision support device according to the embodiment. FIG. 8 is a functional block diagram showing an example of the configuration of a judgment support system according to a modified example. FIG. 9 is a flowchart showing an example of a process performed by a decision support device according to a modified example.

The following describes in detail the embodiments of the judgment support system according to the present disclosure with reference to the drawings. Note that the present disclosure is not limited to the description of the following embodiments. Furthermore, the components in the following embodiments include those that are replaceable and easy for a person skilled in the art, those that are substantially the same, or those that are equivalent. Furthermore, the components described below can be omitted, replaced, or modified in various ways without departing from the gist of the present disclosure. In the following embodiments, components necessary for illustrating the embodiments of the judgment support system according to the present disclosure are described, and other components are omitted. Note that in the following description of the embodiments, the same components are given the same reference numerals, and different components are given different reference numerals.

The judgment assistance system is a system that assists a referee or the like in grasping the positional relationship between a boundary line separating the inside and outside of an area used in a competition and a specified judgment target. The judgment assistance system is used, for example, in a competition venue for ball games such as volleyball, tennis, badminton, etc., when making an in-out judgment as to whether a ball (including a feather ball) that falls on the floor is inside or outside the court. The judgment assistance system may also be used, for example, when making a judgment as to whether an athlete in the above ball games, or in judo, wrestling, sumo, etc., has placed part of his or her body outside the court, outside the arena, outside the sumo ring, etc. In the embodiment, volleyball is described as an example of a ball game, but the present disclosure is not limited to this.

(Embodiment)
First, the configuration of a judgment support system 1 according to an embodiment of the present disclosure will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a plan view that typically illustrates the judgment support system 1 according to the embodiment. Fig. 2 and Fig. 3 are views that illustrate an example of a projected image PI. Fig. 4 is a functional block diagram that illustrates an example of the configuration of the judgment support system 1 according to the embodiment. Fig. 5 is a view that illustrates an example of a captured image of a sphere BL. Fig. 6 is a view that illustrates an example of additional information AD.

As shown in FIG. 1, the judgment assistance system 1 of the embodiment is installed in a competition venue 2 where volleyball is played using spheres BL. The competition venue 2 includes a floor 3, first floor seats 4, second floor seats 5, and a referee's seat 6. The judgment assistance system 1 also includes a competition area 10, a photographing unit 20, a projection unit 30, an input unit 50, a display unit 60, and a judgment assistance device 70.

A court is provided on the floor 3, which is divided by lines. In this embodiment, the entire court is set as a playing area 10 to be used for playing. The playing area 10 is divided by boundary lines 11. In this embodiment, the boundary lines 11 are set on the upper surface of the floor 3 so as to separate the inside and outside of the playing area 10. The boundary lines 11 include side lines 12, 13 and end lines 14, 15. The playing area 10 is divided into two zones 10a, 10b by a center line 16 drawn in the middle. Directly above the center line 16, a net 18 is placed between a pair of posts 17 erected on both ends of the center line 16. The first and second floor seats 4 and 5 are provided around the floor 3. The referee's seat 6 is provided in the first floor seats 4 on one of the long sides.

The photographing unit 20 photographs images including the competition area 10 and boundary line 11 during the competition. The photographing unit 20 outputs the photographed image data to the image recording unit 84 of the memory unit 80 described below. The photographing unit 20 includes a camera. It is sufficient that the photographing unit 20, which photographs at least the boundary line 11, can photograph the light projected by the projection unit 30. Multiple photographing units 20 are provided at each location in the competition venue 2.

In the embodiment, eight imaging units 20 are provided, but this disclosure is not limited to this. In the following, when the eight imaging units 20 are described without distinguishing between them, they will be described as imaging units 20, and when the eight imaging units 20 are described with different reference numerals, imaging units 21 to 28, will be described.

The photographing unit 21 is provided, for example, in the first floor seats 4 on the zone 10b side (upper side in FIG. 1). The photographing unit 21 photographs, for example, the entire sideline 12 on one side (left side in FIG. 1) from outside zone 10b. The photographing unit 22 is provided, for example, in the first floor seats 4 on the zone 10a side (lower side in FIG. 1). The photographing unit 22 photographs, for example, the entire sideline 13 on the other side (right side in FIG. 1) from outside zone 10a.

The camera unit 23 is provided, for example, in the first floor seats 4 on the opposite side of the playing area 10 from the umpire's seat 6 (the left side in FIG. 1). The camera unit 23, for example, photographs the entire end line 14 of zone 10a. The camera unit 24 is provided, for example, in the first floor seats 4 on the opposite side of the playing area 10 from the umpire's seat 6. The camera unit 24, for example, photographs the entire end line 15 of zone 10b.

The photographing unit 25 is provided, for example, in the second floor seats 5 on the side of zone 10a. The photographing unit 25 photographs, for example, the entire sideline 12 from outside zone 10a. The photographing unit 26 is provided, for example, in the second floor seats 5 on the side of zone 10b. The photographing unit 26 photographs, for example, the entire sideline 13 from outside zone 10b.

The camera unit 27 is provided, for example, in the second floor 5 on the side of the judges' seats 6 (the right side in FIG. 1) relative to the competition area 10. The camera unit 27 captures, for example, the entire zone 10a. The camera unit 28 is provided, for example, in the second floor 5 on the side of the judges' seats 6 relative to the competition area 10. The camera unit 28 captures, for example, the entire zone 10b.

Of the photographing units 20, the photographing units 21, 22, 25, and 26 that photograph the sidelines 12 and 13 are each set to have approximately the same angle of view and zoom, etc. Among the photographing units 20, the photographing units 23 and 24 that photograph the endlines 14 and 15 are each set to have approximately the same angle of view and zoom, etc. Among the photographing units 20, the photographing unit 27 that photographs the entire zone 10a and the photographing unit 28 that photographs the entire zone 10b may be installed, for example, on the ceiling of the competition venue 2 so as to photograph from above.

2 and 3, the projection unit 30 projects the projection image PI along the boundary line 11. Here, projecting along the boundary line 11 means projecting directly above an area including the width of the boundary line 11 or a range a predetermined distance away from the width. The projection unit 30 projects the three-dimensional projection image PI along the boundary line 11 in a range directly above the boundary line 11, the width of the boundary line 11, and a predetermined height from the top surface of the floor 3, based on projection information acquired from a projection image generating unit 91 of the judgment assistance device 70 described later. In addition, when projecting the three-dimensional projection image PI, the projection unit 30 may project the three-dimensional projection image PI in a range including the periphery at a predetermined distance from the width of the boundary line 11. The method of projecting the projection image PI by the projection unit 30 may be, for example, a projection method used in projection mapping. In the embodiment, the projection unit 30 includes a projector. The light with which the projection unit 30 projects the projection image PI may be visible light or invisible light.

As shown in FIG. 1, multiple projection units 30 are provided at various locations in the competition venue 2. It is preferable that each projection unit 30 is positioned so that light is not blocked by obstacles such as athletes. For example, if the competition is an indoor competition, the projection unit 30 may be placed on the ceiling or the like so as to project from above.

In the embodiment, twelve projection units 30 are provided, but this disclosure is not limited to this. In the following, when the twelve projection units 30 are described without distinguishing between them, they will be described as projection units 30, and when the twelve projection units 30 are described with different reference numerals, projection units 31 to 42, will be described.

Projection unit 31 is provided, for example, in the second floor seats 5 on the zone 10a side. Projection unit 32 is provided, for example, in the second floor seats 5 on the zone 10b side. Projection unit 33 is provided, for example, in the second floor seats 5 on the opposite side of the competition area 10 from the referee's seats 6. Projection units 31, 32, and 33 project a projection image PI, for example, along one sideline 12 (see FIG. 2).

Projection unit 34 is provided, for example, in the second floor seats 5 on the zone 10a side. Projection unit 35 is provided, for example, in the second floor seats 5 on the zone 10b side. Projection unit 36 is provided, for example, in the second floor seats 5 on the opposite side of the competition area 10 from the referee's seat 6. Projection units 34, 35, and 36 project projection image PI, for example, along the other sideline 13.

Projection unit 37 is provided, for example, in second floor seating 5 on the opposite side of the competition area 10 from the judges' seat 6. Projection unit 38 is provided, for example, in second floor seating 5 on the judges' seat 6 side of the competition area 10. Projection unit 39 is provided, for example, in second floor seating 5 on the zone 10a side. Projection units 37, 38, and 39 project projection image PI, for example, along end line 14 of zone 10a (see FIG. 3).

Projection unit 40 is provided, for example, in the second floor seats 5 on the opposite side of the competition area 10 from the judges' seats 6. Projection unit 41 is provided, for example, in the second floor seats 5 on the judges' seats 6 side of the competition area 10. Projection unit 42 is provided, for example, in the second floor seats 5 on the zone 10b side. Projection units 40, 41 and 42 project projection image PI, for example, along the end line 14 of zone 10a.

The input unit 50 can accept a predetermined input operation for the operator to input information to the imaging unit 20, the projection unit 30, the display unit 60, and the judgment assistance device 70. The input unit 50 is realized, for example, by a physical input device such as a keyboard, button, lever, dial, switch, etc. connected to the judgment assistance device 70 by wire or wirelessly, or a touch panel provided on the display unit 60. The input unit 50 outputs an instruction signal according to the accepted input operation.

The display unit 60 is realized by a display device such as a liquid crystal display (LCD). The display unit 60 displays various information including characters and images. The display unit 60 can display, for example, an image captured by the imaging unit 20. When the display unit 60 includes a touch panel as the input unit 50, it can display an image for input operation for performing a touch operation by the input unit 50. When a touch operation is performed on an area of the surface of the touch panel that corresponds to an image for input operation displayed on the display unit 60, the input unit 50 outputs a predetermined instruction signal including position information of the area where the touch operation was performed.

The judgment support device 70 performs judgment support processing to help a referee or the like who checks the positional relationship between the boundary line 11 and the sphere BL from a captured image to understand the relationship. As judgment support processing, for example, the judgment support device 70 generates a projection image PI to be projected along the boundary line 11, and detects the point in time when the sphere BL overlaps with the projection image PI based on the captured image including the projection image PI. The referee or the like can use the captured images taken before and after the point in time when the sphere BL overlaps with the projection image PI for making a judgment. As shown in FIG. 4, the judgment support device 70 includes a memory unit 80 and a calculation unit 90.

The storage unit 80 is realized by a storage device such as a semiconductor memory such as a RAM (Random Access Memory), a VRAM (Video Random Access Memory), a flash memory, or an SSD (Solid State Drive), a magnetic disk such as a HDD (Hard Disk Drive), an optical disk such as a CD-ROM or a DVD, or online storage.

The memory unit 80 stores various data in the judgment assistance system 1. The memory unit 80 records, for example, data acquired and generated by the judgment assistance system 1. The memory unit 80 stores, for example, various control programs and data used for various control processes. The memory unit 80 is also used as a working area for the calculation unit 90, and temporarily stores the results of calculations performed by the calculation unit 90 in accordance with the various control programs. The memory unit 80 includes a competition area information memory unit 81, a projection image memory unit 82, a sphere information memory unit 83, a video recording unit 84, and an additional information recording unit 85.

The competition area information storage unit 81 stores competition area information indicating information on the competition area 10. The competition area information includes, for example, the position, width, and depth of the competition area 10, and the position, width, and length of the boundary line 11. The positions of the competition area 10 and the boundary line 11 may be expressed as coordinate values relative to a predetermined reference point set in the competition venue 2. The competition area information storage unit 81 can edit the values of various items of the competition area information by operation from the input unit 50. The competition area information storage unit 81 may be capable of storing multiple types of competition area information corresponding to different competition areas 10. In this case, the competition area information storage unit 81 can add, edit, and delete each piece of competition area information by operation from the input unit 50. The competition area information stored in the competition area information storage unit 81 is referenced by the projection image generation unit 91 of the calculation unit 90.

The projection image storage unit 82 stores projection images that indicate image information of the projection image PI to be projected directly onto the boundary line 11 by the projection unit 30. The projection images stored in the projection image storage unit 82 are referenced by the projection image generation unit 91 and the projection determination unit 93 of the calculation unit 90.

The sphere information storage unit 83 stores sphere information indicating image information of a comparison sphere BL for detecting a sphere BL from a captured image indicating a frame image constituting the video data. The sphere information stored in the sphere information storage unit 83 is referenced by the sphere detection unit 92 of the calculation unit 90.

The video recording unit 84 records the video data captured by the photography unit 20 during the competition. The video data includes moving image data consisting of successive frame images. The video recording unit 84 outputs the captured images showing the frame images that make up the video data to the sphere detection unit 92 of the calculation unit 90.

The additional information recording unit 85 records the additional information AD (see FIG. 6) generated by the additional information generating unit 95 of the calculation unit 90. The additional information AD will be described in the section on the additional information generating unit 95. The additional information AD recorded by the additional information recording unit 85 is referenced by the captured image area identifying unit 96 of the calculation unit 90.

The calculation unit 90 is realized by a calculation processing device that is composed of, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ISP (Image Signal Processor), a GPU (Graphics Processing Unit), or a GPGPU (General-Purpose Computing on Graphics Processing Unit). The calculation unit 90 is implemented by one calculation processing device or multiple calculation processing devices.

The calculation unit 90 executes commands included in various control programs stored in the memory unit 80. The calculation unit 90 includes a projection image generation unit 91, a sphere detection unit 92, a projection determination unit 93, a detection information generation unit 94, an additional information generation unit 95, and a captured image area identification unit 96, which are connected by a bus.

The projection image generating unit 91 acquires the competition area information stored in the competition area information storage unit 81. The projection image generating unit 91 acquires the projection image stored in the projection image storage unit 82. The projection image generating unit 91 generates projection information indicating information for projecting the projection image IP along the boundary line 11 based on the competition area information and the projection image. More specifically, the projection image generating unit 91 generates projection information for projecting a three-dimensional projection image IP directly above the boundary line 11 within a range of the width of the boundary line 11 and a predetermined height from the top surface of the boundary line 11. The projection image generating unit 91 outputs the projection information to the projection unit 30. The projection unit 30 projects the projection image IP along the boundary line 11 based on the projection information. Here, the projection information may include information regarding the position and direction in which the projection image is projected. It may also include information regarding the color and pattern of the projected image. It may also include information indicating the type of projection light source suitable for projecting the projection image, such as a halogen light source, an LED light source, or a laser light source.

The sphere detection unit 92 periodically acquires captured images including the boundary line 11 from the video recording unit 84. The sphere detection unit 92 acquires sphere information stored in the sphere information storage unit 83. The sphere detection unit 92 detects the image area of the sphere BL from the captured image based on the sphere information. The sphere detection unit 92 detects the image area of the sphere BL, for example, by extracting the sphere BL from the captured image based on shape information of the sphere BL. The sphere detection unit 92 outputs the captured image and detection information of the image area of the sphere BL to the projection determination unit 93.

The projection determination unit 93 acquires detection information of the captured image and the image area of the sphere BL. The projection determination unit 93 acquires the projection image stored in the projection image storage unit 82. The projection determination unit 93 determines whether or not reflected light of the projection image PI is detected in the image area of the sphere BL based on the projection image.

The example of the captured image shown in FIG. 5 is an image of the sideline 12 captured by the image capture unit 21. FIG. 5 shows a sphere BL falling from the inside to the outside of the playing field 10. In the example shown in FIG. 5, when the sphere BL is in the position shown by the dotted line, the image area of the sphere BL does not overlap with the projected image PI projected directly on the boundary line 11. In this case, the projection determination unit 93 determines that the reflected light of the projected image PI has not been detected in the image area of the sphere BL. When the sphere BL is in the position shown by the solid line, the reflected light of the projected image PI is projected in the image area of the sphere BL. In this case, the projection determination unit 93 determines that the reflected light of the projected image PI has been detected in the image area of the sphere BL. The projection determination unit 93 outputs the determination result to the detection information generation unit 94.

The detection information generating unit 94 generates detection information when the projection determining unit 93 determines that reflected light of the projection image PI has been detected in the image area of the sphere BL. The detection information generating unit 94 outputs the generated detection information to the additional information generating unit 95.

The additional information generating unit 95 generates predetermined additional information AD to be associated with the captured image based on the detection information. The additional information AD includes information for identifying the image capturing units 21-28 and information on the detected time. In the example shown in FIG. 6, the additional information AD includes a file name indicating information for identifying the image capturing units 21-28, a time code indicating the time from the start of recording to the detected time, and a recording date and time indicating the date and time when each additional information AD was recorded. The additional information generating unit 95 outputs the generated additional information AD to the additional information recording unit 85 of the storage unit 80.

The captured image area identifying unit 96 identifies a captured image area in which reflected light of the projected image PI is detected in the image area of the sphere BL based on the additional information AD from a captured image associated with multiple pieces of additional information AD. For example, the captured image area identifying unit 96 causes the captured image area associated with the desired additional information AD to be presented via the display unit 60 in response to an operation by an operator from the input unit 50.

Next, the flow of processing in the judgment support device 70 of the judgment support system 1 according to the embodiment of the present disclosure will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an example of processing by the judgment support device 70 according to the embodiment. The processing shown in FIG. 7 is executed by the judgment support device 70 based on a predetermined control program and data. Note that, before executing the processing shown in FIG. 7, it is assumed that the competition area information, the projection image, and the sphere information are stored in the storage unit 80 by the operator via the input unit 50. It is also assumed that the projection information for projecting the projection image PI along the boundary line 11 has already been generated by the projection image generation unit 91. When the projection unit 30 projects the projection image IP and the image capture unit 20 captures the competition area 10, for example, the judgment support device 70 transitions to step ST111 shown in FIG. 7 and starts processing by accepting a judgment support start operation by the operator.

The judgment support device 70 acquires a captured image (step ST111). Specifically, the sphere detection unit 92 acquires a captured image including the boundary line 11 from the video recording unit 84. The judgment support device 70 proceeds to step ST112.

The judgment assistance device 70 detects the image area of the sphere BL from the captured image (step ST112). Specifically, first, the sphere detection unit 92 acquires the sphere information stored in the sphere information storage unit 83. Next, the sphere detection unit 92 detects the image area of the sphere BL from the captured image based on the sphere information. The sphere detection unit 92 outputs the captured image and the detection information of the image area of the sphere BL to the projection determination unit 93. The judgment assistance device 70 proceeds to step ST113.

The judgment assistance device 70 judges whether or not reflected light of the projection image PI has been detected in the image area of the sphere BL (step ST113). Specifically, first, the projection judgment unit 93 acquires the projection image stored in the projection image storage unit 82. Next, the projection judgment unit 93 judges whether or not reflected light of the projection image PI has been detected in the image area of the sphere BL based on the projection image. The projection judgment unit 93 outputs the judgment result to the detection information generation unit 94.

If reflected light of the projection image PI is detected in the image area of the sphere BL (step ST113; Yes), the judgment support device 70 proceeds to step ST114. If reflected light of the projection image PI is not detected in the image area of the sphere BL (step ST113; No), the judgment support device 70 returns to step ST111 and periodically executes the processes from step ST111 to step ST113.

When reflected light of the projection image PI is detected in the image area of the sphere BL (step ST113; Yes), the judgment support device 70 generates detection information (step ST114). Specifically, when the projection determination unit 93 determines that reflected light of the projection image PI has been detected in the image area of the sphere BL, the detection information generation unit 94 generates detection information. The detection information generation unit 94 outputs the generated detection information to the additional information generation unit 95. The judgment support device 70 proceeds to step ST115.

The judgment support device 70 generates additional information AD (step ST115). Specifically, the additional information generation unit 95 generates predetermined additional information AD to be associated with the captured image based on the detection information. The additional information AD includes information for identifying the imaging units 21 to 28 and information on the time of detection. The additional information generation unit 95 outputs the generated additional information AD to the additional information recording unit 85 of the storage unit 80. The judgment support device 70 proceeds to step ST120.

The judgment assistance device 70 determines whether or not to terminate the judgment assistance system 1 (step ST120). Specifically, the judgment assistance device 70 determines, for example, whether or not an operation to terminate judgment assistance by the operator has been accepted. If the judgment assistance system 1 is to be terminated (step ST120; Yes), the judgment assistance device 70 terminates the processing of the flowchart shown in FIG. 7. If the judgment assistance system 1 is not to be terminated (step ST120; No), the judgment assistance device 70 returns to step ST111 and periodically executes the processing from step ST111 to step ST120.

(Modification)
Next, the configuration of a judgment support system 1A according to a modified example will be described with reference to Fig. 8. Fig. 8 is a functional block diagram showing a configuration example of the judgment support system 1A according to a modified example. Note that in the judgment support system 1A of the modified example, the same components as those in the judgment support system 1 of the embodiment are denoted by the same reference numerals and will not be described as appropriate, and different components will be described.

The judgment support system 1A of the modified example differs from the judgment support system 1 of the embodiment in that it includes a judgment support device 70A instead of the judgment support device 70. The judgment support device 70A differs from the judgment support device 70 of the embodiment in that it includes a memory unit 80A and a calculation unit 90A instead of the memory unit 80 and the calculation unit 90. The memory unit 80A differs from the memory unit 80 of the embodiment in that it further includes a nearby area information memory unit 86A. The calculation unit 90A differs from the calculation unit 90 of the embodiment in that it further includes a sphere position determination unit 97A and a projection control unit 98A.

The neighborhood information storage unit 86A stores neighborhood information indicating information on the neighborhood area indicating the range from the boundary line 11 to a predetermined distance. The neighborhood information includes position information indicating the range of the predetermined neighborhood area including the boundary line 11 within the field of view of the image captured by the imaging unit 20. The range of the neighborhood area may be expressed by coordinate values relative to a predetermined reference point set within the field of view of the captured image. The neighborhood information storage unit 86A can edit the values of various items of the neighborhood information by operation from the input unit 50. The neighborhood information storage unit 86A may be capable of storing multiple types of neighborhood information corresponding to different competition areas 10 and boundary lines 11. In this case, the neighborhood information storage unit 86A can add, edit, and delete each piece of neighborhood information by operation from the input unit 50. The neighborhood information stored in the neighborhood information storage unit 86A is referenced by the sphere position determination unit 97A of the calculation unit 90A.

The sphere position determination unit 97A acquires detection information of the captured image and the image area of the sphere BL. The sphere position determination unit 97A acquires the nearby area information stored in the nearby area information storage unit 86A. The sphere position determination unit 97A determines whether the image area of the sphere BL is within the nearby area based on the nearby area information. The sphere position determination unit 97A outputs the determination result to the projection control unit 98A.

When the sphere position determination unit 97A determines that the image area of the sphere BL is within the nearby area, the projection control unit 98A causes the projection unit 30 to start projecting the projection image PI. The projection control unit 98A also acquires detection information from the detection information generation unit 94. Based on the detection information, the projection control unit 98A causes the projection unit 30 to stop projecting the projection image PI.

Next, the flow of processing in the judgment support device 70A of the judgment support system 1A according to the modified example will be described with reference to FIG. 9. FIG. 9 is a flowchart showing an example of processing by the judgment support device 70A according to the modified example. The processing shown in FIG. 9 is executed by the judgment support device 70A based on a predetermined control program and data. Note that, before executing the processing shown in FIG. 9, it is assumed that the competition area information, the projection image, and the sphere information are stored in the storage unit 80 by the operator via the input unit 50. It is also assumed that the projection information for projecting the projection image PI along the boundary line 11 has already been generated by the projection image generating unit 91. For example, when the photographing unit 20 is photographing the competition area 10, the judgment support device 70A proceeds to step ST201 shown in FIG. 9 and starts processing by accepting, for example, an operation to start judgment support by the operator.

The judgment support device 70A acquires a captured image (step ST201). Specifically, the sphere detection unit 92 acquires a captured image including the boundary line 11 from the video recording unit 84. The judgment support device 70A proceeds to step ST202.

The judgment support device 70A detects the image area of the sphere BL from the captured image (step ST202). Specifically, first, the sphere detection unit 92 acquires the sphere information stored in the sphere information storage unit 83. Next, the sphere detection unit 92 detects the image area of the sphere BL from the captured image based on the sphere information. The sphere detection unit 92 outputs the captured image and the detection information of the image area of the sphere BL to the sphere position determination unit 97A. The judgment support device 70A proceeds to step ST203.

The judgment support device 70A judges whether the image area of the sphere BL is within the nearby area (step ST203). Specifically, first, the sphere position judgment unit 97A acquires the nearby area information stored in the nearby area information storage unit 86A. The sphere position judgment unit 97A judges whether the image area of the sphere BL is within the nearby area based on the nearby area information. The sphere position judgment unit 97A outputs the judgment result to the projection control unit 98A.

If the image area of the sphere BL is within the nearby area (step ST203; Yes), the judgment support device 70A proceeds to step ST210. If the image area of the sphere BL is not within the nearby area (step ST203; No), the judgment support device 70A returns to step ST201 and periodically executes the processes from step ST201 to step ST203.

If the image area of the sphere BL is within the nearby area (step ST203; Yes), the sphere position determination unit 97A starts projecting the projection image PI (step ST210). Specifically, if the sphere position determination unit 97A determines that the image area of the sphere BL is within the nearby area, the projection control unit 98A starts projecting the projection image PI at the projection unit 30. The judgment support device 70A proceeds to step ST211.

The processes of steps ST211, ST212, ST213, ST214, and ST215 are similar to those of steps ST111, ST112, ST113, ST114, and ST115 in the embodiment, and therefore will not be described. However, in step ST215, the detection information generation unit 94 outputs the generated detection information not only to the additional information generation unit 95, but also to the projection control unit 98A. The judgment support device 70A proceeds to step ST216.

The judgment support device 70A stops the projection of the projection image PI (step ST216). Specifically, the projection control unit 98A causes the projection unit 30 to stop projecting the projection image PI based on the detection information. The judgment support device 70A proceeds to step ST220.

The processing of step ST220 is the same as the processing of step ST120 in the embodiment, and therefore description thereof will be omitted. If the judgment support system 1 is to be terminated (step ST220; Yes), the judgment support device 70A terminates the processing of the flowchart shown in FIG. 9. If the judgment support system 1 is not to be terminated (step ST220; No), the judgment support device 70A returns to step ST201 and periodically executes the processing from step ST201 to step ST220.

As described above, the judgment assistance system 1, 1A generates a projection image PI to be projected along the boundary line 11, and detects the point in time when the sphere BL overlaps with the projection image PI based on a captured image including the projection image PI. The referee or the like judges the positional relationship between the boundary line 11 and the sphere BL based on the captured images before and after the point in time when the sphere BL overlaps with the projection image PI. This makes it possible to shorten the time required to judge the positional relationship between the boundary line 11 and the sphere BL. Furthermore, because the relationship between the sphere BL and the projection image PI can be confirmed from the captured image, the positional relationship between the boundary line 11 and the sphere BL can be easily grasped.

The components of the illustrated judgment assistance systems 1 and 1A are functional concepts and do not necessarily have to be physically configured as illustrated. In other words, the specific form of the judgment assistance system is not limited to that shown in the figures, and all or part of it may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of the judgment assistance system.

The configuration of the judgment assistance system is realized, for example, as software, by a program loaded into memory. In the above embodiment, these functional blocks are described as being realized by the cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms, by hardware alone, software alone, or a combination of both.

The above-mentioned components include those that a person skilled in the art would easily imagine and those that are substantially the same. Furthermore, the above-mentioned configurations can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the configurations are possible without departing from the gist of this disclosure.

For example, the light used by the projection unit 30 to project the projection image PI along the boundary line 11 may be laser light. In other words, the projection unit 30 may be a laser illuminator. Note that the projection image PI does not need to be formed as an actual visible image directly on the boundary line 11, but it is sufficient that the reflected light of the projection image PI can be confirmed in the image area of the sphere BL when the sphere BL passes through the range where the projection image PI is projected.

In the embodiment, the object to be judged is the sphere BL, the boundary lines are the sidelines 12, 13 and the end lines 14, 15, and an in/out judgement of whether the point where the sphere BL lands is inside or outside the court has been described, but the present invention is not limited to this. For example, if the sport is volleyball, the object to be judged may be the player's foot and the boundary line may be the center line or attack line within the court, or the object to be judged may be the player's hand and the boundary line may be a net. Furthermore, the judgment assistance system is not limited to ball games and may be used for line judgments in track and field, in-field and out-of-field judgements in martial arts, etc.

1, 1A Judgment support system 10 Competition area 11 Boundary line 20 Shooting unit 30 Projection unit 50 Input unit 60 Display unit 70, 70A Judgment support device 80, 80A Memory unit 81 Competition area information memory unit 82 Projected image memory unit 83 Sphere information memory unit 84 Video recording unit 85 Additional information recording unit 86A Nearby area information memory unit 90, 90A Calculation unit 91 Projected image generation unit 92 Sphere detection unit (object detection unit)
93 Projection determination unit 94 Detection information generation unit 95 Additional information generation unit 96 Captured image area specification unit 97A Sphere position determination unit (object position determination unit)
98A Projection control unit BL Sphere (object to be determined)
PI Projected image AD Additional information

Claims (5)

A storage unit that stores competition area information including the width of a boundary line that divides the competition area used in the competition;
a projection image generating unit that generates projection information for projecting a projection image onto an area including the width of the boundary line or a range that is a predetermined distance away from the width of the boundary line based on the competition area information ;
an object detection unit that detects an image area of an object to be determined from the captured image including the boundary line;
a projection determination unit that determines whether or not the projection image is detected in an image area of the object to be determined;
a detection information generating unit that generates detection information when the projection image is detected in an image area of the object to be determined;
A judgment assistance system comprising: A projection unit projects the projection information generated by the projection image generation unit onto an area including a width of the boundary line or a range that is a predetermined distance away from the width of the boundary line.
The decision support system according to claim 1 . the projection image generating unit generates projection information for projecting a three-dimensional projection image in a range from the boundary line to a predetermined height directly above a region including a width of the boundary line or a range separated by a predetermined distance from the width of the boundary line,
The decision support system according to claim 1 or 2 . an additional information generating unit that generates, when the detection information is generated, predetermined additional information to be associated with the photographed image;
The decision support system according to claim 1 . A step of storing competition area information including a width of a boundary line that divides a competition area to be used in a competition;
generating projection information based on the competition area information for projecting a projection image onto an area including the width of the boundary line or a range that is a predetermined distance away from the width ;
detecting an image area of the object to be determined from the captured image including the boundary line;
determining whether the projection image is detected in an image area of the object to be determined;
generating detection information when the projection image is detected in an image area of the object to be determined;
A judgment assistance method comprising:

Images