Detecting Post-Migration Data Loss

About 3% of our member centers have experienced total migration failures in which data from birth years 2015, 2016, and 2017 appears to have been imported into eNICQ 6 in a complete and correct state, but on investigation, proves to have most data items missing on most records.  Other more minor discrepancies in data are also possible and can be resolved.  It is essential to screen for any data loss after migration, before starting to enter 2018 records in eNICQ 6.

Data loss during migration that goes undetected or unreported can become more difficult to resolve later on if it goes undetected or unreported.  Some data loss scenarios are easy to solve, others may require more extensive interventions.  In some cases, it is necessary to redo the migration, so it is desirable to address any data loss prior to starting 2018 data entry to prevent having to rework/re-enter 2018 records as well.


Early Indicators of Possible Data Loss

The following issues may be an early warning on data loss.

  • Did an exception happen during data migration?  It is important to collect the full wording of any exception that happens during migration.  Typically, exceptions will appear in a separate popup with a details panel that can be copied and pasted.  Please collect this information if possible.
  • Did the application time out during data migration?  It is possible that the migration routine may take a while to process but it should normally come to conclusion with a results message, or an error message.  If it fails to complete or reports a time out, this should be 


Reviewing Your Data for Data Loss

Please follow the instructions below following migration.

  • Open eNICQ 6 and perform a Batch Validation.  This option is available under Settings (the gear icon) -> Batch Validate on the Patient Log.
  • Review your status codes on the Patient Log.  Are there any surprises, such as one or more records that are showing as "Incomplete" that you previously considered to be "Complete?"
    • If you see one or more unexpected Incompletes, what data items are incomplete?  Is it the same data item consistently blank?  Is the data item related to Still Hospitalized Status?  Is there another recognizable pattern, such as problems with infection items?
    • Please report to Vermont Oxford Network that you have unexpected Incomplete items and what data items are showing as incomplete.
      • When problems are limited to one or two data items in a recognizable pattern, Vermont Oxford Network is usually able to assist you with resolution through manual or scripted work arounds.  
  • Open several historical records that were imported for 2015, 2016, 2017.  Open both Complete and Incomplete records (if you have examples of both.)
    • Are any records marked as "Complete" showing missing data items when you open them?  Be sure to scroll down from top to bottom through several "Complete" records of each historical birth year to confirm that there is no data loss.
    • If you have found "Complete" records that are missing almost all data items after the initial eligibility items (after gestational age,) please notify Vermont Oxford Network of the data loss immediately.  Please review more of your imported records to determine if it is just a few records missing data or if it is all imported records missing data.
      • Stop all work in the eNICQ 6 application at this time.  Resume work only after the issue has been resolved.
      • Are there any other centers co-hosted with this center on the same eNICQ 6 database?  If so, stop all work in those centers as well and perform these same screening steps to check for data loss.
      • Complete data loss of all items after Gestational Age on all or most records is a serious issue and may require re-import of the data or rebuilding of the database.  Please work with Vermont Oxford Network technical support until the issue is resolved.


Exception Messages Associated with Data Loss

The most commonly associated exception for post-migration data loss has been "The given key was not present in the dictionary."  This message is usually reported to Vermont Oxford Network by the SubmissionFailed subroutine in instances of near-complete data loss.  The exception begins to surface post-migration when the data set is first handled by the eNICQ 6 application.