INC1015328967: Frequently Observing 500 Read timed out error from Domestic Payment Submission API

Description

From: Janardhana R Anumala <Janardhana.R.Anumala@aexp.com>
Sent: 01 November 2019 14:09
To: BarclaysOpenBankingQueries <BarclaysOpenBankingQueries@barclayscorp.com>
Cc: Ramnath K Krishnan <Ramnath.K.Krishnan@aexp.com>; Vignesh Veeraiyan <Vignesh.Veeraiyan@aexp.com>; Margaret H Ryan1 <Margaret.H.Ryan1@aexp.com>; Amit Gupta <Amit.Gupta10@aexp.com>; Nikhil Gokuldas Bhutada <Nikhil.G.Bhutada@aexp.com>; Prashant Kadam <Prashant.Kadam1@aexp.com>
Subject: Re: INC1015328967: Frequently Observing 500 Read timed out error from Domestic Payment Submission API

Hi Barclays Team,

Again we observed Read timed out error in payment submission and funds moved from customer account.. we resubmitted with same x-idempotency-key to avoid customer impact but no luck, got same error in retry , kindly look into this and let us know the best way to avoid customer impact on these errors.. ..

domestic_payment_request_headers=Content-Type = application/json;Accept = application/json;x-idempotency-key = 75185983-56d0-402f-9f1b-1d4a91791d1d;x-fapi-financial-id = 0015800000jfAW1AAM;x-fapi-interaction-id = 75185983-56d0-402f-9f1b-1d4a91791d1d;x-fapi-customer-ip-address = 193.32.30.50;x-jws-signature = eyJiNjQiOmZhbHNlLCJodHRwOlwvXC9vcGVuYmFua2luZy5vcmcudWtcL2lhdCI6MTU3MjYwODc3OSwiaHR0cDpcL1wvb3BlbmJhbmtpbmcub3JnLnVrXC90YW4iOiJvcGVuYmFua2luZy5vcmcudWsiLCJjcml0IjpbImI2NCIsImh0dHA6XC9cL29wZW5iYW5raW5nLm9yZy51a1wvaWF0IiwiaHR0cDpcL1wvb3BlbmJhbmtpbmcub3JnLnVrXC90YW4iLCJodHRwOlwvXC9vcGVuYmFua2luZy5vcmcudWtcL2lzcyJdLCJraWQiOiJrMjdTbk1JMFF0a2NHX1I0MVRUakJlQWRjdW8iLCJodHRwOlwvXC9vcGVuYmFua2luZy5vcmcudWtcL2lzcyI6IjAwMTU4MDAwMDFIUVFzaUFBSFwvRUpYWmpCT0JRc21yOFhHUXUzWEk3NCIsImFsZyI6IlBTMjU2In0..GkeXdz9EDMCgv1ygfxrXHVD_N-Qs8vDmnW2wuLx3ULAPIJxMUSzD6EV07Qx2YEnb9ZxA3JQiJQhgJk-zT___hoV9JLQvBHLcDZYfyx9wbmW7agx3UN2nf6fDJ4XlTH_EHPk6NZofgPIWB5Fg8boIgzDH4WeBj-V9bYnMED5BEuOHRQf23nqKrBq95W2YkrTm8Xlu9_G9P4l--PpLCzbO7RGUZElenKbWkWMXvzaJH6e0-uBppeOVadMn7Ue0NwfV7tWopOn3RK6bbnlKj_8RUzh4rAk0l7MR7Yqd8eegGSXfgvHndw0Mn45yTMh3r7qpTmYY75pwuRu7PZO75Rp6xQ;,

domestic_payment_response=500 {"errors":[{"id":"55ea1bf8-4fc3-4f92-bea5-785e4cb69757","code":"ZP_ISE_001","title":"Internal Server Error","detail":"Read timed out"}]},

domestic_payment_consent_response_headers=Content-Type = application/json;charset=ISO-8859-1;Content-Length = 136;Connection = keep-alive;Date = Fri, 01 Nov 2019 11:46:19 GMT;Server = Apache;X-Frame-Options = DENY;X-Content-Type-Options = nosniff;X-Download-Options = noopen;Referrer-Header = no-referrer;Strict-Transport-Security = max-age=31536000; includeSubDomains;X-Xss-Protection = 1; mode=block;Cache-Control = no-cache, no-store, must-revalidate;X-Application-Context = zuul-gatewayrod:82fec3350af96f735b41409e7d48f30a;x-jws-signature = eyJraWQiOiJaMlNkcmtqX0JIeGxGVHNlRngtYXBTQ2hDYm8iLCJhbGciOiJSUzI1NiIsImN0eSI6ImFwcGxpY2F0aW9uL2pzb24iLCJiNjQiOmZhbHNlLCJjcml0IjpbImI2NCIsImh0dHA6Ly9vcGVuYmFua2luZy5vcmcudWsvaWF0IiwiaHR0cDovL29wZW5iYW5raW5nLm9yZy51ay90YW4iLCJodHRwOi8vb3BlbmJhbmtpbmcub3JnLnVrL2lzcyJdLCJ0eXAiOiJKT1NFIiwiaHR0cDovL29wZW5iYW5raW5nLm9yZy51ay9pYXQiOjE1NzI2MDg3ODksImh0dHA6Ly9vcGVuYmFua2luZy5vcmcudWsvdGFuIjoib3BlbmJhbmtpbmcub3JnLnVrIiwiaHR0cDovL29wZW5iYW5raW5nLm9yZy51ay9pc3MiOiJDTj0xVk83R09hV3k0NUhzQVJ1djhhZmZsLE9VPTAwMTU4MDAwMDBqZkFXMUFBTSxPPU9wZW5CYW5raW5nLEM9R0IifQ..YKXgOsh7kvwoOd2QZn0vWpm4b_qaCifk-6OafJ_c01pc9ihmzg6tANkSMzqjNrzHw1tjzhpXjFnuUWQWoT_QlJ21UfQgIKAvTfATx19jII5PgjrCJltO8R1chZaqsyqUr1wBRdIRrZfJWIBM0m87sMJowK067IzJvq8BVlKUqYQxOauV43ne9AesuYZhw8oSV-zR8jCMGr3sJpGjx-DDkqLUjl88GTbjanE4-Mv78oaH-aWR7jD2niy6e2MOPDuYSg6W8pj4QRfTAExlIdtQoAEh5zA6KY92zzox6o9JHOaF6cjJnQSKyyXJjc-XaFrEAPUE4BUVwKk8N4kutxWDkQ;Set-Cookie = 09cc08b965e567968f2b5dc4f6ea9dcf=1502c88a0469753cf272b10dee55ff96;path=/;HttpOnly;Secure;,

Regards
Jana
Amex PISP
Janardhana.r.anumala@aexp.com

From: "BarclaysOpenBankingQueries@barclays.com" <BarclaysOpenBankingQueries@barclays.com>
Date: Tuesday, October 15, 2019 at 8:49 AM
To: Amit Gupta <Amit.Gupta10@aexp.com>, "BarclaysOpenBankingQueries@barclays.com" <BarclaysOpenBankingQueries@barclays.com>
Cc: Ramnath K Krishnan <Ramnath.K.Krishnan@aexp.com>, Vignesh Veeraiyan <Vignesh.Veeraiyan@aexp.com>, Janardhana R Anumala <Janardhana.R.Anumala@aexp.com>, Margaret H Ryan1 <Margaret.H.Ryan1@aexp.com>
Subject: INC1015328967: Frequently Observing 500 Read timed out error from Domestic Payment Submission API

Hello Amit,

Thank you for your email, we have passed the information onto our support teams who have identified the issue.

We are currently employing a fix and shall let you know once we have successfully implemented this.

Apologies for the inconvenience caused.

Kind regards,

OB Support

Restricted - External
From: Amit Gupta <Amit.Gupta10@aexp.com>
Sent: 15 October 2019 15:36
To: BarclaysOpenBankingQueries <BarclaysOpenBankingQueries@barclayscorp.com>
Cc: Ramnath K Krishnan <Ramnath.K.Krishnan@aexp.com>; Vignesh Veeraiyan <Vignesh.Veeraiyan@aexp.com>; Janardhana R Anumala <Janardhana.R.Anumala@aexp.com>; Margaret H Ryan1 <Margaret.H.Ryan1@aexp.com>
Subject: Frequently Observing 500 Read timed out error from Domestic Payment Submission API

This mail originated from outside our organisation - Amit.Gupta10@aexp.com
Hi Team,

We are frequently observing timeout issues from Barclay’s APIs. Today morning we have received couple of timeout errors from Domestic Payment Submission API.

PSU shown transaction error on merchant page but still the amount is debited from their account. We have analyzed from our side and we have observed that we are receiving timeout issue from domestic payment submission API. We were unable to notify the merchant about these Payments.

Kindly advise how we can address this timeout issue without impacting PSU and how we can track this kind of transactions?

PFB the request and response for failure transaction.

API URL : https://clicktime.symantec.com/3BvhaVezmWeVZzRATUBJ4rM6H2?u=https%3A%2F%2Ftelesto.api.barclays%2Fopen-banking%2Fv3.1%2Fpisp%2Fdomestic-payments

Request 1 :

Timestamp : 2019-10-15T10:23:00.200+0000
x-fapi-interaction-id : c573d947-a08e-47cb-a627-0ca7937be16b
client-id : 301e3c2daf1f4233a40fbd4545090b32

Response :

500 {"errors":[{"id":"37a1860b-cb85-45c2-9557-b1a6c388ff12","code":"ZP_ISE_001","title":"Internal Server Error","detail":"Read timed out"}]}

Request 2 :

Timestamp : 2019-10-15T10:30:55.694+0000
x-fapi-interaction-id : 36d7a5b6-51f4-482e-bdd6-32e363acd625
client-id : 301e3c2daf1f4233a40fbd4545090b32

Response :

500 {"errors":[{"id":"57059325-06a4-46fb-87af-6568d1c72e83","code":"ZP_ISE_001","title":"Internal Server Error","detail":"Read timed out"}]}

Thanks,
Amit

From: "BarclaysOpenBankingQueries@barclays.com" <BarclaysOpenBankingQueries@barclays.com>
Date: Tuesday, October 15, 2019 at 5:18 AM
To: Amit Gupta <Amit.Gupta10@aexp.com>, "BarclaysOpenBankingQueries@barclays.com" <BarclaysOpenBankingQueries@barclays.com>
Cc: Ramnath K Krishnan <Ramnath.K.Krishnan@aexp.com>, Vignesh Veeraiyan <Vignesh.Veeraiyan@aexp.com>, Janardhana R Anumala <Janardhana.R.Anumala@aexp.com>
Subject: RE: Observed 500 Server error from Domestic Payment Consent API

Hello Amit,

We have identified the issue and can now see successful responses for this channel. Can we ask you to retry please and confirm that this no longer an issue?

Kind regards,

OB Support

Restricted - External
From: Amit Gupta <Amit.Gupta10@aexp.com>
Sent: 15 October 2019 06:56
To: BarclaysOpenBankingQueries <BarclaysOpenBankingQueries@barclayscorp.com>
Cc: Ramnath K Krishnan <Ramnath.K.Krishnan@aexp.com>; Vignesh Veeraiyan <Vignesh.Veeraiyan@aexp.com>; Janardhana R Anumala <Janardhana.R.Anumala@aexp.com>
Subject: Observed 500 Server error from Domestic Payment Consent API

This mail originated from outside our organisation - Amit.Gupta10@aexp.com

Hi Team,

We have observed 500 Internal server error from Domestic payment consent API , can you please have a look at the below issue and let us know if you need any more information about below failed request.

Request :

Timestamp : 2019-10-14T14:32:33.663+0000
x-fapi-interaction-id : 15451649-fa5a-455b-b5d2-8ba49a288c32
client-id : 301e3c2daf1f4233a40fbd4545090b32

Response :

500 {\"errors\":[{\"id\":\"a88e5336-b467-46ff-b64b-43e174f63826\",\"code\":\"ZP_ISE_001\",\"title\":\"Internal Server Error\",\"detail\":\"Read timed out\"}]}

Thanks,
Amit

American Express made the following annotations
________________________________________

Technical Impact

None

Workaround

None

Resolution Notes

None

Impact Assessment

None

Status

Assignee

Unassigned

Reporter

Service Desk

Reference

None

Service Desk Reference

OBSD-11913

ASPSP

Query Type

None

Created (Original)

Nov 04, 2019, 8:20 AM

TPP Impact

High

TPP

OB Environment

None

Business Impact Severity

None

Share

Yes
Configure