This article explains how to access and configure the Clarity Data Import Tool (DIT).
Accessing the Data Import Tool
To access the Data Import Tool (DIT), your Clarity user account must be assigned a role that has been configured to allow access. To use the DIT for importing data, click the Launchpad icon and select Data Import.
Once the user has accessed the Data Import Tool, there are a number of sections on the right sidebar.
The Home screen provides the user an area to upload files to be imported into the Clarity system. The DIT will accept HMIS CSV or XML data in format version FY2020. The user can drag and drop a file onto the DIT interface, or click Select File to provide the traditional file upload process using a file explorer.
The Settings screen provides the user the ability to configure a number of items which determine how the import process interacts with the data being imported.
Client matching is in place to allow the user to control how the imported data may merge with existing records in the Clarity system. When continually importing data, it is important to ensure that you are not creating duplicate records and have measures in place to match incoming records with existing records and update them as needed, which the Client Matching settings provide. Client matching is imperative to ensure data quality resulting in accurate reporting.
Client matching leverages a number of client fields including a client's First Name, Last Name, DOB, and SSN. The options available are described below.
- Regular Matching - When enabled, the import process will take the client records being imported and compare them with client records already in the Clarity system. If a partial match is made on a client's First Name, Last Name, DOB, and SSN, it will be considered a match. In this case, the existing client record will be updated with the client data in the file if there are any differences.
- Full Matching -When enabled, the import process will take the client records being imported and compare them with client records already in the Clarity system. An exact match will need to be made on a client's First Name, Last Name, DOB, and SSN. If an exact match is not made, the client record and associated data will not be imported, and no client data will be created or updated.
- No Matching - When enabled, client matching will not occur during import which will cause a new client records to be created in Clarity regardless if the client already exists in the system. This is typically used during the introduction of new clients into a clean database.
Allow Custom Fields
This toggle setting enables the ability to use custom fields in the import process. When this setting is turned off, the Custom Fields advanced setting will not appear in the right sidebar.
Validation Errors Limit
This setting allows a limit to be configured on the number of reported validation errors reported. In the event a user is importing a very large file, data validation can take an extended amount of time. Typically when validation errors occur, they are caused by a small number of distinct errors, however can be repeated several hundred times
When a limit is set, the validation process will stop when the configured number of errors has been met allowing the user to resolve those issues throughout the file without waiting for the entire dataset to be validated.
Rejected Clients Log
The Rejected Client Log screen provides the user with information on clients who have been rejected in previous imports when Full Client Matching is enabled. The user has the ability to filter the log contents leveraging a number of column name fields. A description of those filterable fields and others are described in the table below.
|File ID||Displays the ID of the file the client record was included in when rejected|
|Client ID||Displays the ID of the client record that was rejected within the file|
|Client Unique Identifier||Displays the Unique Identifier of the client if available|
|Client SSN||Displays the Social Security Number belonging to the client that was rejected|
|Client First Name||Displays the first name of the client rejected|
|Client Last Name||Displays the last name of the client rejected|
|Client DOB||Displays the date of birth of the client rejected|
|Rejected Reason||Displays the reason the client was rejected|
The Custom Fields functionality allows the user to import data that expands beyond the HMIS XML schema data elements. For example, if a legacy system contains a data element that does not exist in Clarity, that field can be created in Clarity and configured for use on this screen. You must have the Allow Custom Fields setting enabled from the Settings screen in order for this section to appear.
For more information on creating customer data fields in Clarity, please review the Creating & Editing Custom Data Fields help center article.
Note: The Custom Fields functionality only works with HMIS XML imports. HMIS CSV is not currently supported.
The Custom Fields screen is broken into three sections
- Edit Client Records
- Edit Enrollment Fields
- Publish Custom fields
Edit Client Fields
Clicking on the Edit Clients Fields button provides an area to work with custom fields specifically related to client records.
At the top of the screen there is a toggle to Enable All Fields. This adds all client related data fields that are not included in the XML schema. Use caution with this setting if you have a large amount of custom fields in your site, as all of the fields will be active in the schema.
There is also a Client Fields search area and dropdown to specify whether the user is searching for Core Clarity client fields that come stock in the system, or Custom fields that have been added to the system by an administrator.
When searching, auto-suggest will display potential matches in the system. Clicking on an existing field will add it to to the Client fields List section. To remove a custom field, click the delete icon that appear when you hover over the target field.
Edit Enrollment Fields
Very similar to the Edit Client Fields screen above, the Edit Enrollment Fields is specifically intended to work with fields related to client enrollments --Entries, Status Assessments, Annual Assessments, and Exits.
Publish Custom Fields
To ensure that the newly configured custom fields are in place and allow for validation to occur on imported files, you MUST publish any changes to the two areas above to ensure successful validation. Clicking the Publish Custom Fields button updates the XML schema stored within the Clarity system to include the newly added fields.
When a file is imported containing the custom fields, they are validated against this stored and updated XML schema to ensure the user is importing the data the Clarity system is configured to accept.