AISP API v2.0
In order to mimic pagination, we request transactions in 2 week chunks of time. This allows us to bypass the documented 10MB response size limit. However it appears for some requests, the fromBookingDateTime and toBookingDateTime are being ignored, resulting in the full payload for the consented transaction range being returned.
This results in duplicate transactions being processed and presumably comes up against the stated 10MB response limit for busy accounts.
We notice from the self link above that query parameters appear to have been applied twice internally (fromBookingDateTime and toBookingDateTime are blank). We compared the malformed responses with normally filtered ones, and normally filtered responses contained a slightly differently constructed "Self" link
x-fapi-interaction-id1: e893e650-867b-4266-83f3-bc7ea7b90f34 (fromBookingDateTime=2019-07-16T00:00:00Z&toBookingDateTime=2019-07-31T23:59:59Z)
x-fapi-interaction-id2: 9ceb8815-0a13-46ed-a3cd-63aef105d8da (fromBookingDateTime=2019-05-13T00:00:00Z&toBookingDateTime=2019-05-28T23:59:59Z)
Very High. Besides the obvious additional processing, we're having to suspend support temporarily while adding additional safeguards to detect and flag this scenario. Of more concern than duplicated data is the fact that this is potentially happening with "busy" accounts and triggering the 500 errors for payload size, making AISP resource fetching impossible for these PSUs.