Data Migration

How to do a data migration with the Data Migration Portal.

🤔 What is the Data Migration Portal?

The Data Migration Portal helps you with any data migrations you want to make into Fast Track CRM. With the help of the portal, you're now completely independent in making data migrations. With no dependencies on Fast Track, you can work at your own pace and according to your own schedule. Should you however require any assistance, please reach out to your Integration Manager at Fast Track.
​

🔎 Where to find the Data Migration Portal

In order to find and access the Data Migration Portal, your user needs to have the Integration Management Admin role assigned.

How to navigate:

  1. From the Fast Track CRM side menu, click on the Settings-button
  2. Click on Integrations
  3. Select Data Migration
Navigating to the Data Migration Portal
Navigating to the Data Migration Portal
​

⚙️ How to do a Data Migration

Step 1: Planning

  1. Decide what tables and columns you want to migrate.
  2. Here you need to understand what Segment Fields will be affected by your migration.
  3. Note that the Segment Fields in Fast Track CRM are called table_name-field_name. For example the Segment Field with the name payment_activity-deposit_count refers to the deposit_count column from the payment_activity table.
  4. It is important to note that if a field is not migrated, then using it in Segmentation can result in undesired behaviour due to the fact that players could have null values for the non-migrated fields.
  5. You must indicate whether a table should be truncated prior to the data migration. ⚠‍ Selecting this option will clear the table prior to filling it with the migrated data. This option should not be selected if you only intend to migrate the data for a subset of your players.
  6. Once you have selected your tables, you should download template CSVs for the selected tables. You will need these for uploading files in the next step.

Keep an eye out 👀‍​

  1. When planning, keep in mind the granularity of the tables that you are migrating. If you are migrating payment_events, every row in your uploaded CSV file should correspond to a status change in a payment transaction. If you are migrating casino_activity , every row will correspond to a single player so the data should represent an aggregate summary for that player.
  2. Be mindful of when you should truncate tables;
    1. If you are migrating a complete history of events, it is safe to truncate the table.
    2. If you are migrating an aggregate table, you should only truncate if you plan on including all players in your migration.
    3. If you are migrating a selection of columns from an aggregate table, you should not truncate.
​

Step 2: Upload

  1. You will need to save the migration before proceeding to upload your test data.
  2. For each selected table in the previous stage, you should upload your files in the upload section. You can upload multiple files for the same table. We will take care of consolidating them into a single file for you.
  3. The files need to have the same structure as the templates downloaded in the previous step.
  4. Proceed to mapping; here you need to map the uploaded files to the tables selected for migration. If you map a file that does not have the correct structure as per the downloaded template, you will get an error when trying to map it to the table.

Keep an eye out 👀‍​

  1. When extracting data files for your upload, pay extra attention to timestamp/date columns. All migrated timestamp/date columns should be in YYYY-MM-DD format and in UTC time.
  2. When uploading files for the dry run, we advise you to use a big chunk (if not all) of your players. This ensures that any potential issues in the data are caught in the dry run, before starting the full data migration.
​

Step 3: Dry Run

  1. Just click the button.
  2. Wait for the migration to complete.
  3. When your migration is complete, go to the QA stage.

Step 4: QA

  1. doHere you will get a summary of the migrated data.
  2. You will also get presented with a set of validation checks that were done and whether those validations passed or not.
  3. Follow each step in the QA and verify that everything looks good.
  4. Download the reconciliation files and check that the data that was migrated looks good.
  5. When you are happy with the results, sign off the dry run and proceed to do the full run.

Keep an eye out 👀‍​

  1. If anything goes wrong in the dry run, for example, you notice inconsistencies in the data during the QA process, it is safe to abort your migration and start a new one. This since no production data is modified during the dry run.
  2. Use the QA process of the dry run to check and verify that your migrated data is perfect. If you are not convinced that your data looks right, reach out to your Fast Track Integration Manager for assistance.
​

Step 5: Full Run

The full run of the migration follows the same flow as the dry run. What is important to note here is that in order to do the full run, Fast Track will need to pause the processing of real-time events and the firing of CRM Activities. This is done to ensure data consistency and to make sure that no Activities are firing based on stale Segmentation data.

Step 5.1: Pause Queues

  1. The first step in executing the full run of the migration is pausing the queues.
  2. When you click the pause queues button, you will be asked for confirmation. Upon confirmation, the system will pause the processing of messages from the queues.
  3. You will be provided with the exact timestamp of the last processed message before the queues were paused. It is important to take note of this timestamp.
  4. When doing data exports, all fields included in the migration should be calculated as at the provided timestamp. When the real-time processing is resumed, the Fast Track system will continue aggregating real-time events from that timestamp onwards, ensuring consistency of the data.

Step 5.2: Upload Files for Full Run

  1. The structure of the files remains the same as the files uploaded in the dry run.
  2. You should now upload files for each of the migrated tables. We remind you, that the data in the files should be aggregated as at the timestamp provided in the previous step.
  3. Once you have uploaded the files for all of your tables, proceed to the next step.

Step 5.3: Run

  1. Start the migration process by clicking the run button.
  2. Wait for the migration to complete - this may take some time, depending on the volume of data that is being migrated.
  3. When the data migration has been completed, proceed to the Quality Assurance step.

Step 5.4: QA

  1. The QA process is similar to the verification process that was done in the dry run.
  2. At this point, you should ensure that all data is correct.
  3. If you sign off the migration, the Fast Track services will resume and processing of messages will continue from the previously provided timestamp onwards.
​ℹ‍You should see the following banner once the migration full-run is completed:
​
​

Step 5.5: Monitoring of Services

  1. After the data migration has been completed the Fast Track services will resume. At this point, you should head over to the service portal and check that your Fast Track system looks healthy.
  2. You should note that the messages that were in the queue will continue to be processed - a build-up of messages in the followings queues is expected since no messages were being processed throughout the duration of the migration:
    1. integration.api
    2. activitymanager.timeinstance
    3. activitymanager.lf-timeinstance
    4. activitymanager.sdtinstance
    5. singularity
    6. singularity-time
  3. Verify that messages are being processed and the number of messages on the queues is going down - you should be given an estimated time when the queue will be fully processed and the system is back in real-time.
  4. If messages are not being processed or something does not look right, reach out to your Fast Track Integration Manager to get help.

Keep an eye out 👀‍​

  1. If you have started a full run of your data migration and for some reason you would like to stop the migration, you should reach out to your Fast Track Integration Manager for assistance.
  2. If you notice something wrong with your data during the QA process, you can roll back the migration, by clicking on Rollback. This will revert your production data to the state it was before the migration started. Alternatively, you can finalise your migration and re-run another one.
  3. It is important to verify that your services are back up and running following your data migration. If you think that something is wrong with your Fast Track services, reach out to your Fast Track Integration Manager for assistance.
​
​