Data Extracts

Overview

Data extracts services help Envestnet | Yodlee customers (cobrands) retrieve their consumers' data in an efficient and secure way. These services provide incremental payload data that is inserted, updated, or deleted in the system for a defined duration.

Implementation Methods

There are two ways to implement data extracts:

Data Updates Event Triggering Actions

Actions that will generate DATA_UPDATES event during the polling or notification methods are:

  • A successful or failed action to add a provider account through API or add an account through FastLink.
  • A successful or failed action to update/refresh a provider account through API, edit credentials or refresh an account through FastLink, or refresh an account through an application.
  • Deleting a provider account through API or an application.
  • Updating details of an aggregated account or a manual account through API or an application, for example, updating the account nickname.
  • Adding a manual account through API or FastLink.
  • Deleting an aggregated account or a manual account through API or an application.
  • Updating an aggregated transaction through API or an application, for example, updating the transaction category.
  • Cache/auto refreshes
  • Reconciliation process

Supported Entities

Entities that are supported in data extracts services are as follows:

  • Provider account
  • Account
  • Transactions
  • Holdings

Get User Data Service

On receiving the data extracts notification, invoke the Get User Data service to retrieve the changed data. The Yodlee system will only consider those users as active for whom the customer is retrieving the data.

Payload Details

The deleted records of an entity are indicated by the isDeleted attribute as true Get User Data service response. The response of the Get User Data service includes the following entities:

Entities Response Description
transaction The entity will contain all the attributes returned in the GET /transactions endpoint along with the isDeleted attribute.
holding The entity will contain all the attributes returned in the GET /holdings endpoint along with the isDeleted attribute.
providerAccount The entity will contain all the attributes returned in the GET /providerAccounts endpoint along with the isDeleted attribute.
account The entity will contain all the attributes returned in the GET /accounts endpoint along with the isDeleted and isSelectedForAggregation (applies only to FastLink 4 customers with account summary selection screen) attributes.

Payload Processing Recommendations

Your payload processing and database updates should rely on the following logic:

  • Provider accounts processing –
    • The providerAccountId is a unique identifier that can be used to perform add and update operations.
    • The isDeleted flag will be returned as true for deleted records. Using the isDeleted flag, the providerAccounts that are deleted from the Yodlee database can be identified and the same can be deleted from your database. Ensure that you delete all the child records such as transactions, holdings, and accounts associated with the deleted providerAccount.
  • Accounts processing –
    • The accountId and providerAccountId together will be the unique identifier of an account that can be used to perform add and update operations.
    • The isDeleted flag will be returned as true for deleted records. Using the isDeleted flag, the accounts that are deleted from the Yodlee database can be identified and the same can be deleted from your database. Ensure that you delete the transactions and holdings associated with the deleted accountId.
    • For FastLink 4 unified flow customers, the isSelectedForAggregation flag will indicate if the account is selected or not on the aggregation account summary selection screen. Using the flag, unselected accounts (i.e., isSelectedForAggregation=false) can be identified, and you can delete them from your database. Ensure that you delete all transactions and holdings associated with the unselected account IDs. This process only applies to active verification-only accounts in the database, as these accounts (i.e., isSelectedForAggregation=false) are only associated with the verification flow but not with the aggregation flow.
  • Transactions processing –
    • The transactionId for an aggregated account will be unique. The add and update operations can be performed depending on the transactionId and isManual fields.
    • The isDeleted flag will be returned as true for deleted records. Using the isDeleted flag, the transactions that are deleted from the Yodlee database can be identified and the same can be deleted from your database.
    • During the update account process, the pending transactions are deleted from the Yodlee database and a new transactions is created. The deleted transaction and the new transaction will be provided in the data extracts
  • Holdings processing –
    • The holdingId is the unique identifier of a holding. As holdings are always updated during the investment or insurance account refreshes, only the active holding records (i.e., no deleted records) will be provided in the data extracts response.
    • We recommend that you update, insert, and delete by comparing the provided holdings with your database records.

Process the get user data payload in the following sequential way:

  1. Process provider accounts information
  2. Process accounts information
  3. After processing provider accounts and user accounts, transactions and holdings can be processed in parallel.

Notes:

  • As Yodlee does not store any historical information about accounts, transactions, or statements, the response of the Get User Data service will have information about the latest action that has happened on the resource. For example, if an account is updated twice at 5 minutes intervals, the get user data API call for the first 5 minutes will return the latest account information.
  • Do not call the Get User Data service with a time difference of more than 60 minutes.
  • Do not call the Get Data Extracts Events service with a time difference of more than 60 minutes.
  • Do not call the Get Data Extracts Events service with dates that are less than 7 days.

Event Payload Information

The following JSON payload will be sent to the callback URL and the content type will be application/json:

"event": {
	"notificationId": "64b7ed1a-1530523285",
	"info": "DATA_UPDATES.USER_DATA",
	"data": {
		"userCount": 1,
		"fromDate": "2017-11-10T10:18:44Z",
		"toDate": "2017-11-10T11:18:43Z",
		"userData": [{
			"user": {
				"loginName": "YSL1484052178554"
			},
			"links": [{
				"methodType": "GET",
				"rel": "getUserData",
				"href": "dataExtracts/userData?fromDate=2017-11-10T10:18:44Z&toDate=2017-11-10T11:18:43Z&loginName=YSL1484052178554"
			}]
		}]
	}
}
Event Attributes Name Details
info The info attribute value will be DATA_UPDATES.USER_DATA for the data updates event.
data The data node is the payload the customer receives and is used by the developer to take relevant actions. The data attribute varies from event to event.
userCount The count of users for whom the data has changed
fromDate The dateTime from when the notification is generated
toDate The dateTime till when the notification is generated
userData The userData node contains the user and the link information
user The user node is an entity that contains information about the users.
loginName The loginName of the user for whom the data is changed
links The links node is an entity that contains link-related information
methodType The type of the method to invoke the endpoint
notificationId Unique identifier for notification.
rel The relationship detail of the service to be invoked
href The URL to be appended to the base URL to invoke the related service.

Frequently Asked Questions

  1. What happens when a user attempt to update an account and the attempt fails? Will a notification be triggered?
    Yes. The accounts with the failed status get returned in the get user data service.
  1. What happens when there is no change to account data during an update?
    A notification is still sent to indicate that the account has been updated even if there are no data changes. This is to let the customer know that the balance is up-to-date.
  1. What happens when the update fails due to login errors? A failure notification is sent.
    The get user data will provide the provider account and all the accounts belong to the provider account, though the update has failed.
  1. Is it possible to receive multiple notifications for one provider account update? Yes.
    Let us deal this question with an example of a provider account which has 4 accounts.
    Assumptions: Notification interval = 10 min. Total time taken for the update = 12 mins.
    10th Min: 3 accounts got updated. 1 account is in progress. Notification gets generated.
    Get user data will return the provide account and 3 accounts
    12th Min: 1 account that was in progress got updated. Notification gets generated
    Get user data will return the provider account and 1 account.
  1. Do you return all the accounts that belong to the provider account all the time?
    All accounts for which updates are attempted are returned.
  1. What happens if the provider account update is only partially successful?
    Accounts that are successfully updated and failed accounts are returned in the get user data.
  1. If the user has two accounts and only one account is updated during auto refreshes, will both the accounts be returned in the get user data service?
    No, only the account for which the update has been attempted will be returned in the get user data service.