Hello,
We are designing an ingestion pipeline that loads Data Exports into our data warehouse, and we would like to confirm how delta exports should be handled before we finalize it.
Specifically:
- What time window does each delta export cover? Does it include only the changes since the previous delta, or a longer period?
- We are evaluating the paid tier, where deltas run every 30 minutes. If each delta covers a longer window than that, consecutive deltas would contain overlapping records. Is that the expected behavior?
- Given the above, what loading strategy do you recommend: appending each delta, or merging it into the existing tables?
Thank you in advance.