Importing Data into Insight Magnet
This document is intended for technical audience assisting the preparation and management of data import into Insight Magnet.
Choosing a format
Insight magnet supports the following formats for importing data. Please evaluate the choices based on your particular situation. Once decided it is best to standardize all your data imports on one choice.
The following applies to all data imports. Considerations for preparing the files are outlined. The actual data content is discussed further based on the specific requirement.
A native Excel file is supported. The “.xlsx” format if preferred but we can also import the “.xls” format (Excel 97-2004).
When using this format, consider the following -
- Make sure you have only one worksheet in the document. If you have multiple items to import please use a different file for each list.
- Remove all formatting from the spreadsheet.
- Remove all macros and links to other spreadsheets.
- We prefer the file to include only values as compared to calculated formulas.
- Remove blank cells/rows at the end of the worksheet.
- Ensure that you save the file with UTF-8 or UTF-16 encoding
CSV - (Comma Separated Values)
Insight Magnet can import a properly formatted CSV files. The file can be comma delimited (with an .csv extension) or TAB delimited (with a .txt extension).
When using this format, consider the following.
- CSV files contain one record per line. Ensure that the end of line character is correctly rendered and one record does not span more than one lines.
- The first row of the file should contain column headers.
- All text strings in the data should be quoted. Please check to ensure that valid quotes that are part of the string are properly escaped.
- Remove blank lines at the end of the worksheet.
- Ensure that you save the file with UTF-8 or UTF-16 encoding.
- Checking the CSV format is particularly important if you are exporting data from Excel or other desktop tool. Check the created text file in an text editor to ensure that commas, new line characters, and, escapes are properly rendered. In addition, if using a combination of Mac version of Excel, select MS-DOS Comma Separated (.csv) as the export format.
You may choose to Base64 encode text fields for additional security in transmission of the files. Please document the encoded fields as you would need to tell the Insight Magnet uploader this information so that the data can be decoded before loading into the system.
If you are planning for an automated loading of periodic data (such as nightly audience list for sending out satisfaction surveys), it is important to create and test the file from the source system. If you are using a CRM tool, ensure that the tool creates the export and you use the file as-is to test the upload process. Saving the system generated export on a PC or Mac can lead to differences in encoding and can lead to errors when moving from testing to production.
Planning update frequency
As you prepare any import file, plan on using it as a template for loading the same data again. Insight Magnet uses the same file for first time imports and updates.
The ID fields
The ID fields are unique identifiers of the data element. In database terminology, it could map to a primary key of the data master table. It is important that an ID field is supplied as Insight Magnet relies on this field to decide if the incoming data is a new record or an update to an existing record. Whenever possible, this should be the unique ID from the parent system from where the data is exported. If the ID is not available, then you will have to create one in the import by using a simple sequence. While doing so, you will need to make sure that duplicate records are not part of the import or if there are duplicate records they have the same unique ID for the data element.
Silent Attributes is a mapping for information that you already know about the entity. For example, for a User, you may have demographic and geographical data available. You may already know the Postal or ZIP code, length of relationship as a customer, gender, age group etc. This information is typically loaded into Insight Magnet and Insight Magnet can use this in the targeting of surveys and analysis of the data.
Silent Attributes are available for Users, Customers, Cases (when a survey invitation is triggered by a sales or service transaction), Employees and Reviewers. Up to 12 silent attributes are available in the default configurations for each entity.
All silent attribute values are treated as “strings”. Once loaded you can specify the sort order for displaying the ordered list of silent attributes in the system.
Audience Pool/Users/Survey Recipients
This is a list of users that you want to send the surveys out to. In a B2C environment, each of the recipient will be associated with a Company/client. Insight Magnet refers to the individual answering the survey as “User” and the Company he is associated with as “Customer”. If you are in a direct to consumer market, you may not have Customer data for a User.
Definitions for User data
|ID||This is an identifier from your source system that uniquely identifies a user. Insight Magnet tracks this field to determine if the record specified should be a new record or update to an existing record.|
|CustomerID||In a B2C environment, the unique ID of the Customer this user belongs to. This field is used as a mapping key to decide new records or updates to the user.|
|Email for the user. Unique for each user. An email address cannot be shared by two or more users.|
|First Name||First Name|
|Last Name||Last Name|
|Display Name||If specified, will be used as the user salutation.|
|Silent Attributes||Silent Attributes for the user|
If you are initiating a survey based on a case (a transaction, a support case, completion of an order etc.) then you can provide the list of cases as a separate data feed. In that feed, you would only reference the user using the User ID. For a one time survey, if a case needs to be tracked as a reference point for the feedback to be collected, then the case attributes can be provided in the same row as the user information in the user upload file.
This is a list of customers and is applicable only in the B2C scenario. Customer is a business entity and several users map to a customer. Please note the difference between a “Customer” and a “User” as described earlier.
Definitions for Customer data
|ID||This is an identifier from your source system that uniquely identifies a Customer. Insight Magnet tracks this field to determine if the record specified should be a new record or update to an existing record. This field is also referenced as CustomerID in the User load definition|
|Name||Name of the customer|
|Silent Attributes||Silent Attributes for the Customer|
This is a list of cases that form the basis of triggering survey invitations. This is typically applicable in “transactional surveys” where new shipments, closed support cases etc. are loaded nightly into Insight Magnet and corresponding surveys are sent out to the corresponding users. Note that when using case data it is possible for a single user to get multiple surveys when no governing conditions are set. For example, if a user placed 3 order and all the orders were shipped on the same day, the case data file will contain 3 records for 3 shipped orders. In the absence of any governing conditions, 3 surveys will be mailed to the same user. Governing conditions are set in the tool, but the condition logic often relies on the content of the Case data. Please make sure that the case data has the fields needed and referenced by the governing logic.
Definitions for Case data
|ID||This is an identifier from your source system that uniquely identifies a Customer. Insight Magnet tracks this field to determine if the record specified should be a new record or update to an xisting record. This field is also referenced as CustomerID in the User load definition|
|Silent Attributes||Silent Attributes for the Case|
For Talent Magnet, employee data is loaded using the format specified below. The same format is used for first time load and updates.
Definitions for Employee data
|ID||This is an identifier from your HR system that uniquely identifies an employee. Insight Magnet tracks this field to determine if the record specified should be a new record or update to an existing record.|
|ManagerID||The unique identifier for the employee’s manager. If the employee does not have a manager, this field can be blank. You have the option of providing an ID or the email of the manager.|
|Email for the employee. Unique for each employe. An email address cannot be shared by two or more employees.|
|First Name||First Name|
|Last Name||Last Name|
|Display Name||If specified, will be used as the employee salutation|
|Silent Attributes||Silent Attributes for the employee|