Агрегирование по таблице событий на основе периодов временного окна, настроенных в другой таблице

Question

У меня есть три таблицы: UpEvent, DownEvent и AnalysisWindow

UpEvent:
up_event_id | event_date            |  EventMetric
1              2015-01-01T06:00:00       54
2              2015-01-01T07:30:00       76

DownEvent:
down_event_id | event_date          |  EventMetric
1              2015-01-01T06:46:00         22
2              2015-01-01T07:33:00         34

AnalysisWindow:
window_id |        win_start           |    win_end
1              2015-01-01T00:00:00       2015-01-01T04:00:00
2              2015-01-01T00:00:00       2015-01-01T08:00:00
.
.

Я хочу выполнить анализ в каждом окне AnalysisW, чтобы агрегировать события UpEvent и DownEvent, которые произошли между определенным окном.

Таким образом, для каждой записи AnalysisWindow я получу 1 строку:

WinStart             |  WinEnd               |   TotalUpEvents  |  TotalDownEvents
2015-01-01T00:00:00    2015-01-01T04:00:00         0                  0
2015-01-01T00:00:00    2015-01-01T08:00:00         2                  2

Моей первой мыслью было сделать что-то вроде

select win.win_start, 
       win.win_end, 
       count(ue.*), 
       sum(ue.EventMetric) 
from AnalysisWindow win
left join UpEvent ue on (ue.event_date between win.win_start and win.win_end)

Что, очевидно, не работает.

Я неправильно подхожу к этой проблеме? Я хочу сделать оконный анализ таблиц в различных окнах, которые я настраиваю, и получить 1 совокупную запись на окно

Answer 1

Ниже для BigQuery Standard SQL (и на самом деле работает!)

#standardSQL
WITH ue_win AS ( 
  SELECT 
    window_id, COUNT(1) TotalUpEvents  
  FROM `project.dataset.AnalysisWindow` win
  CROSS JOIN `project.dataset.UpEvent` ue
  WHERE ue.event_date BETWEEN win.win_start AND win.win_end
  GROUP BY window_id
), de_win AS ( 
  SELECT 
    window_id, COUNT(1) TotalDownEvents  
  FROM `project.dataset.AnalysisWindow` win
  CROSS JOIN `project.dataset.DownEvent` de
  WHERE de.event_date BETWEEN win.win_start AND win.win_end
  GROUP BY window_id
)
SELECT 
  window_id, win_start, win_end, 
  IFNULL(TotalUpEvents, 0) TotalUpEvents, 
  IFNULL(TotalDownEvents, 0) TotalDownEvents
FROM `project.dataset.AnalysisWindow` win
LEFT JOIN ue_win USING(window_id)
LEFT JOIN de_win USING(window_id)

Answer 2

Один метод использует коррелированные подзапросы:

select aw.*,
       (select count(*)
        from UpEvent ue
        where ue.event_date between aw.win_start and aw.win_end)
       ) as ups,
       (select count(*)
        from DownEvent de
        where de.event_date between aw.win_start and aw.win_end)
       ) as downs
from AnalysisWindow aw;

Вышеуказанное работает, по крайней мере, когда сформулировано как:

with UpEvent as (
      select 1 as up_event_id, '2015-01-01T06:00:00' as event_date, 54 as EventMetric union all
      select 2, '2015-01-01T07:30:00', 76
     ),
     DownEvent as (
      select 1 as down_event_id, '2015-01-01T06:46:00' as event_date, 22 as EventMetric union all
      select 2, '2015-01-01T07:33:00', 34
     ),
     AnalysisWindow as (
      select 1 as window_id , '2015-01-01T00:00:00' as win_start, '2015-01-01T04:00:00' as win_end union all
      select 2, '2015-01-01T00:00:00', '2015-01-01T08:00:00'
     )
select aw.*,
       (select count(*)
        from UpEvent ue
        where ue.event_date between aw.win_start and aw.win_end
       ) as ups,
       (select count(*)
        from DownEvent de
        where de.event_date between aw.win_start and aw.win_end
       ) as downs
from AnalysisWindow aw;

Альтернативой является использование union all:

 ud as (
  select event_date, 1 as ups, 0 as downs from upevent
  union all
  select event_date, 0 as ups, 1 as downs from downevent
 )
select aw.window_id, aw.win_start, aw.win_end, sum(ups), sum(downs)
from AnalysisWindow aw join
     ud
     ON ud.event_date between aw.win_start and aw.win_end
group by aw.window_id, aw.win_start, aw.win_end
union all
select aw.window_id, aw.win_start, aw.win_end, 0, 0
from AnalysisWindow aw
where not exists (select 1 from ud where ud.event_date between aw.win_start and aw.win_end)