Data Sources
Explains how to manage data sources for communications.
Using data in your communications adds a level of personalization that helps your customers feel connected and catered for, increasing the likelihood of them taking positive action on your message. EngageOne™ Communicate supports multiple data import methods and data file types.
The most common method of data import is by simple file upload. For file uploads, support formats are CSV, JSON, XLS, XLSX or XML.
For a series of short video clips on Data Sources, see Help videos: Data.
Select Data. The following screen is displayed, which shows any existing data sources:
To add a new data source, click Add Data Source. The following screen is displayed:
Select the data source type, which can be one of the following:
- Local File - Browse for a file or drag the file across.
- Spectrum - Enter your Spectrum OnDemand credentials and then click Connect to view the datasets.
- Compose - Specify a template that was created in EngageOne™ Compose.
If you select Local File, then select a file, the following screen is displayed:
The Data Standard dropdown allows you to select a locale. This setting determines how the data is interpreted, specifically the formatting of date/time and numeric values. For more information, see Formatting variables.
While a data source is being imported, EngageOne™ Communicate returns you to the main Data Sources screen with a progress bar showing the progress of the upload. For example:
While the upload continues, you can do other things with Communicate. For example, you could upload another data source; the two uploads will progress in parallel. Clicking the X to the right of the progress bar allows you to cancel the upload (unless it is very close to completion).
Typical upload times (based on benchmark tests) are as follows:
Number of columns of data | File size in MB | Upload time in seconds |
---|---|---|
10 | 1.1 | 51 |
10 | 11.1 | 290 |
20 | 2.0 | 55 |
20 | 21.0 | 330 |
30 | 3.0 | 55 |
30 | 30.0 | 339 |
Once the data is imported, click the data source in the list, then View Data to see the data in detail.
Data Schema
Click Data Schema to view the schema. The following screen is displayed:
In this tab, you can do the following:
- Edit a field name.
- Check the Hide box against any fields that you want to hide. Hidden fields do not appear under View Data and cannot be used in any communication using this data source.
- Click the Data standard option to see a dropdown that allows you to select a locale. This setting determines how the data is interpreted, specifically the formatting of date/time and numeric values. See Formatting variables for more information.
- Check the Include in Reporting CSV option against all fields that you want to appear in the Reporting CSV file (see Reporting). Any fields without this box checked will not be included in the Reporting CSV file. Note that you should not check this box for hidden fields. See below for a list of points to consider when including a field for reporting.
When selecting a field for inclusion in the Reporting CSV file, note the following:
- Nested fields (fields with arrays and other fields inside them) are currently not supported. You will not be able to select such fields for reporting.
- Field names must be unique. Having more than one field with the same name included in reporting may cause data to overlap between fields, or not be present at all.
- Video reporting (see Statistics for a Video Communication) does not currently support fields with a full stop (.) inside them (such as first.example). Including such fields may cause the fields values to be empty.
- Field names are case-insensitive; the field with the name "ID" is treated the same way as the field with name "id".
Local File
The following types of file are supported: CSV, JSON, XLS, XLSX or XML.
CSV files
If you import data from a CSV file:
- The first row must be the field names, separated by commas. All spaces are replaced by underscores.
- Each record must be on a separate line within the file.
If the imported CSV file has blank columns, Communicate will automatically remove them and close up the gaps. For example, the following CSV file has blank columns (denoted by adjacent commas in the first row):
name,surname,,,country,,,testcolumn,,,
Joe,Bloggs,test1,test2,test3,test4
Jane,Smith,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19
This CSV file would be tidied up on input to look like this:
Id,name,surname,country,testcolumn
0d789cef-99xy,Jane,Smith,3,6
0c5678ab-88ab,Joe,Bloggs,test3,none
JSON files
[
{
"firstname": "John",
"lastname": "Smith",
"email": "john.smith@mycompany.com",
"order_number": "5555",
"delivery_address": [
"Appartment 234",
"High Road North",
"Some Town"
],
"order_items": [
{
"description":"Complete Puppy Food",
"price":"3.49",
"qty":"2",
"total":"6.98"
},
{
"description":"Chunky Dog Treats",
"price":"0.99",
"qty":"5",
"total":"4.95"
}
],
"subtotal": "27.55",
"salestax": "6.88",
"total": "34.43",
"est_del_date":"September 12 2019"
}
]
This example has two nested arrays:delivery_address
is an example of a string array. EngageOne™ Communicate imports the key of the array as an array. This means that in the metadata generated for the JSON data, the key of that array is marked as array.order_items
is an example of an object array. EngageOne™ Communicate imports the key of the array as an array, For all the fields for the JSON objects inside the array, the keys are marked as either string or array, as appropriate.
For information about how to include array data into repeating table rows, see Repeating data in tables.
XLSX files
The first row must contain the field names.
If the imported XLSX file has blanks in the first row, EngageOne™ Communicate will automatically close up the gaps. For example, the following XLSX file has blank field names for columns C and E:
This XLSX file would be tidied up on input to look like this:
Unique Identifier
When a data source is uploaded for the first time, the entire dataset is scanned for unique fields or columns. The first unique field or column is selected as the unique identifier and stored against the data source.
When more data sources are uploaded, the upload process confirms that the unique identifier field or column is present and also contains unique data.
Spectrum
To retrieve up-to-date information from third-party sources for your communications, use EngageOne™ Communicate's connection with Spectrum OnDemand. For more information, see Spectrum OnDemand.
Enter your Spectrum OnDemand credentials, then click Connect to view the datasets.
Compose
The EngageOne™ Compose datasource functionality allows you to use the same data with a Compose and Communicate template.
Before you can use Compose templates in Communicate:
- Your Communicate team must be integrated with Compose. This can be enabled by contacting Support. If your team is not integrated, Compose related functions will not be displayed.
- You must have published some Compose assets to Communicate. For details of how to do this, see EngageOne™ Compose/Communicate: Integrated Features User's Guide.
To specify a Compose template as a data source, click Compose, select the appropriate template from the dropdown list, then click Add Data Source.
Using Data Sources for Video communications
For video communications, Data Sources can have a flat data structure or a nested data structure.
This example shows a flat data structure:
{
var1: 1,
var2: 2
}
This example shows a nested data structure:
{
node1: {
var1: 1,
var2: 2
}
}
Downloading Data Sources
There may be occasions when you need to download a data source from EngageOne™Communicate to a file. To do this, on the Data Sources screen, click the download button () on the right of the screen against the appropriate data source.
EngageOne™Communicate will create and download a file of a type that depends on the original data source (the type of file that was imported). The rules are as follows:
Original file type | Downloaded file type |
---|---|
XLS | CSV |
XLSX | CSV |
CSV | CSV |
JSON | JSON |
XML | JSON |