REPOSITORY CODING · MEDIUM
Distinguish counter wraps from device resets
A device counter crossing its modulus is reported as a reset, losing the wrapped increment. Implement the specified inclusive wrap thresholds and retain ordinary monotonic differences, reset behavior and per-session first-reading rules.
Data EngineerDevice counter session laboratory
Work through a real codebase.
Choose an environment, investigate the existing implementation and use the tests to check your change. Sign in to open a workspace; execution and Ono usage follow your plan’s allowances.
Task requirements and environments
PySpark · 55 minute session
- Sessionize offline device counter observations using real PySpark transformations. Each device/timestamp pair identifies one reading. Exact redelivery is harmless, conflicting counter values at that timestamp are invalid, and observations may arrive out of order. Counters lie in [0,modulus).
- Within each device, a monotonic counter contributes its difference. A decrease is a wrap only when previous*4 >= modulus*3 and current*4 <= modulus; then contribute modulus-previous+current. Other decreases are resets and contribute current. A first reading contributes zero.
- Start a new session only when the elapsed timestamp gap is strictly greater than maxGap. The first reading of every new session contributes zero even when a previous counter exists. Sum increments within each session, count unique observations, and report activeSeconds=end-start.
- Apply minSamples inclusively after complete session aggregation. Duplicate deliveries cannot satisfy that threshold. Return sessions sorted by device then start, with start, end, samples, increment and activeSeconds. The local Spark session is reused; no external telemetry source, platform account or database is accessed.