max |direct - evidently| over 60 checks: 0.0000
max |direct - evidently| chi-square over 40 checks: 0.000000
100 random half splits of the videos, 400 resamples for thresholds, 200 batches per split

batch = 10 videos, reference about 85 videos
column          thr median  default alarm  calibrated alarm
timeofday            0.274            90%                5%
weather              0.352            96%                6%
traffic              0.225            77%                6%
vehicles             0.216            60%                5%
sign                 0.102             5%                5%
traffic_light        0.127            12%                5%
pedestrian           0.080             2%                8%
opposite_lane        0.223            37%                5%
at least one of 8 columns over its threshold: default 100%, calibrated 35%, calibrated at 1-0.05/8 10%
same splits: frames chi-square / one row per video chi-square / one row per video two-sample chi-square
  timeofday     100% /   9% /   5%
  weather       100% /  13% /   7%
dark videos in batch -> detection on timeofday: default / calibrated / per video chi-square / per video two-sample, then mean distance, sd, threshold
  1/10 (10%): 99% / 0% / 1% / 0%   d=0.179 sd=0.020 thr=0.274
  3/10 (30%): 93% / 14% / 28% / 17%   d=0.190 sd=0.060 thr=0.274
  6/10 (60%): 100% / 95% / 100% / 100%   d=0.366 sd=0.038 thr=0.274

batch = 20 videos, reference about 85 videos
column          thr median  default alarm  calibrated alarm
timeofday            0.248            65%                4%
weather              0.295            81%                5%
traffic              0.182            51%                5%
vehicles             0.162            33%                5%
sign                 0.075             1%                6%
traffic_light        0.094             4%                7%
pedestrian           0.073             0%                8%
opposite_lane        0.163            20%                4%
at least one of 8 columns over its threshold: default 97%, calibrated 35%, calibrated at 1-0.05/8 11%
same splits: frames chi-square / one row per video chi-square / one row per video two-sample chi-square
  timeofday      98% /  14% /   6%
  weather       100% /  14% /   5%
dark videos in batch -> detection on timeofday: default / calibrated / per video chi-square / per video two-sample, then mean distance, sd, threshold
  3/20 (15%): 42% / 2% / 7% / 2%   d=0.104 sd=0.060 thr=0.249
  6/20 (30%): 96% / 9% / 62% / 27%   d=0.175 sd=0.031 thr=0.249
  10/20 (50%): 100% / 90% / 100% / 100%   d=0.303 sd=0.021 thr=0.249