Deduplicate telemetry while excluding threshold-equal sessions

Review the independently proposed patch below, including its unchanged callers. Redelivered observations inflate session counts, while sessions exactly at minSamples are discarded. Repair deduplication before window functions and inclusive filtering after full session aggregation.

Review focus: Data Engineer

src/repositories/readings.py

BeforeProposed

+1−1

1111

1212def load(spark, readings):

1313 frame = local_frame(spark, readings, SCHEMA)

14 return frame

14 return frame.dropDuplicates(["device", "at"])

Your review

0 / 10,000 characters · A verdict needs supporting evidence.

30 more characters to give a verdict