Experiment record

SwitchBot five-minute Raw collection: diagnosing minute-04 gaps and reducing ScriptLock contention

A completed verification of a recurring SwitchBot five-minute Raw gap pattern concentrated near minute 04 of each hour. After moving API acquisition before persistence locking and shortening the shared ScriptLock scope, post-change Raw contained all 34 expected minute-04 observations and all 411 expected five-minute cycles.

Summary

AIEL-2026-0007 investigated a recurring five-minute collection failure in which missing SwitchBot observations were concentrated near minute 04 of each hour. Here, Raw means the individual five-minute observation rows retained before any daily aggregation.

The same Google Apps Script project contained two relevant processing paths: the SwitchBot five-minute collector and a separate gas-data refresh path. Both used the same project ScriptLock, a mutual-exclusion lock shared by executions of that Apps Script project. In the retained pre-change source, SwitchBotPower tried to acquire that lock before calling the SwitchBot API and could return after a 20-second timeout without recording an observation. The gas-data refresh path could hold the same lock across a longer processing section, creating a contention path between otherwise separate jobs.

The collector was changed so that SwitchBot API acquisition occurs before persistence locking, the acquisition time becomes the observation timestamp, and the lock is held only around the short write section. The gas-data refresh path was also changed so that it no longer holds the shared lock across its whole refresh operation.

Post-change validation used the five-minute Raw only. Over approximately 34 hours, minute-04 acquisition was 34/34 = 100% and all expected five-minute cycles were present at 411/411. No missing cycle or interval longer than seven minutes was found.

One duplicate observation occurred about 38.5 seconds after the preceding observation. Existing duplicate protection prevented the later observation from being counted twice, and the abnormal-interval cross-check found no recurrence pattern.

Background

Before the change, the SwitchBot path was effectively:

scheduled invocation → acquire shared ScriptLock → SwitchBot API acquisition → write Raw

Because the separate gas-data refresh path used the same project lock, a longer lock hold there could overlap a SwitchBot invocation. If the SwitchBot path could not obtain the shared lock within 20 seconds, the invocation could end before the API call. In that case there was no observation available to save later, so the Raw series contained a silent gap.

For periodic collectors, this distinction matters: persistence contention should not become an acquisition failure when external observation can be performed independently. Acquiring and timestamping the external observation first lets later storage contention be handled as a persistence problem rather than silently suppressing the observation itself.

The source establishes this failure mechanism directly. The attribution of the concentrated minute-04 pattern to long-held shared ScriptLock contention is classified as INFERRED, high confidence, because no direct lock trace was retained for every individual historical missing slot. The inference is supported by the verified mechanism and the disappearance of the pattern after the lock-scope redesign.

Methods

1. Review the pre-change Apps Script control flow and identify the lock-before-observation early-return path.
2. Move SwitchBot API acquisition outside the lock, fix observation time at API acquisition, and keep the lock only around persistence. Reduce the separate gas-data refresh path’s lock scope to its write section as well.
3. Preserve existing five-minute Raw rows and continue normal collection without rewriting history.
4. Validate the post-change Raw for expected five-minute cycles, minute-04 observations, missing cycles, duplicate intervals, and intervals longer than seven minutes.

Daily aggregates were not used for the completion decision, because the test was specifically about whether the primary five-minute observations themselves were present after the collector change.

Experiment Log

Pre-change control flow

The retained SwitchBotPower logic acquired ScriptLock before SwitchBot API acquisition. If the lock was not obtained within 20 seconds, the invocation could return without a Raw observation. A separate gas-data refresh path in the same Apps Script project used the same shared lock across a longer section of its processing.

Collector change

The revised SwitchBot path became:

scheduled invocation → SwitchBot API acquisition + acquisition timestamp → short persistence lock → Raw write

The separate gas-data refresh path also stopped holding the same shared lock across the whole refresh operation and retained locking only for its write section.

Post-change Raw verification

The duplicate occurred about 38.5 seconds after the preceding observation. Existing duplicate protection prevented it from affecting the accumulated result, and it did not form a recurring pattern.

Conclusion

The pre-change source verifies a failure mechanism in which shared ScriptLock contention could terminate a scheduled SwitchBot collection before API acquisition and leave no Raw observation. The redesign removed API acquisition from that lock dependency, preserved acquisition time as observation time, and shortened the shared lock scope on both processing paths that used it.

In the approximately 34-hour post-change Raw window, minute-04 acquisition was 34/34 and total five-minute coverage was 411/411, with zero missing cycles and zero intervals longer than seven minutes. The only duplicate was an isolated event prevented from affecting the accumulated result by the existing duplicate defense.

The verified failure mechanism and the matching pre/post behavior support long-held shared ScriptLock contention as the primary cause of the targeted minute-04 gaps. Because individual historical gaps do not have retained direct lock traces, that causal attribution remains INFERRED rather than directly observed. The repair is VERIFIED effective for the tested failure pattern and validation window.

Evidence summary

The Evidence States for the information obtained in this Experiment are as follows.

See Evidence State for the shared definitions.

Operational implications

For a periodic Apps Script collector, a persistence lock should not determine whether an external observation is acquired when those stages can be separated. Acquiring and timestamping the external observation first, then applying a short persistence lock, reduces the chance that storage contention becomes a silent acquisition gap.

This result is bounded to the tested architecture and failure pattern; it does not imply that every future missing sample must be lock-related.

Machine-readable experiment record

A machine-readable canonical record of this Experiment is published as JSON.

AIEL-2026-0007 experiment.json

Related experiments

AIEL-2026-0001 used an independent five-minute Google Apps Script collection path while investigating SwitchBot OpenAPI stale-status behavior. AIEL-2026-0007 addresses a different failure layer: collector-side missingness caused by observation/persistence lock coupling.