ITSI Version upgrade from 4.4.x to 4.7.x
Planning to upgrade your ITSI to mitigate the products End of Support? Are you facing KVStore issues during upgrade? If yes, this guide will provide you information on how to perform an ITSI upgrade with the KVStore failure.
This guide takes example of ITSI upgrade from version 4.4.x to 4.7.x. In this example, KVStore is rolling back to v4.6.0 from v4.7.2 whereas the updated_itsi_kvstore_version at 4.6.0.
Pre-upgrade actions on the SHC:
- Increased kvstore max_size_per_batch_save_mb = 750MB
- Assessed the KVStore and SHC health and performance baselines
Upgrade steps:
- Check the search head cluster status.
Command: splunk show shcluster-status - Disable the Rules Engine.
- Extract the ITSI installation package on deployer.
Directory: /opt/splunk/etc/shcluster/apps/ - Take backup of /opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/migration_handlers_4_7_0.py
- Edit the original file.
File: /opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/migration_handlers_4_7_0.py
and go to line 236,
dataSources = gt_definition.get(‘dataSources’)
edit that line such that it becomes:
dataSources = gt_definition.get(‘dataSources’, {}) - Save the changes in file.
- Run the following command from SH from CLI
Command:
curl -k -u admin -X POST https://localhost:8089/servicesNS/nobody/SA-ITOA/storage/collections/data/itsi_migration_queue –data ‘{
“skip_pre_checks”: [“KVSIZECHK”, “RECHK”],
“is_migration_triggered”: true,
“skip_local_failure”: true,
}’ \
–header “Content-Type: application/json”
- Get the status again making sure the migration is running now:
Command: curl -k -u admin -X GET https://localhost:8089/servicesNS/nobody/SA-ITOA/migration/info
- Let’s go to the Splunk web UI, but don’t go to ITSI app.
Instead, we go to the main Search app and let’s run this search manually:
index=_internal “itsi.migration” UI source=”*/splunk/var/log/splunk/itsi_migration_queue.log” sourcetype=itsi_internal_log
- When you see this message with the above search:
Data transformation has completed, saving back to KV store…
Then do another search:
index=_internal “itsi.migration” source=”*/splunk/var/log/splunk/itsi_migration_queue.log” sourcetype=itsi_internal_log
When you see the following message (the log will show 4.7.1, not 4.7.2):
Migration bulk save to KV store successfully completed, updating the version to 4.7.1.
- On the master node, place a copy of SA-IndexCreation in $SPLUNK_HOME/etc/master-apps/
- Use splunk apply cluster-bundle to push Index Creation app to peer nodes.
- Upgrade SA-ITSI-Licensechecker and SA-UserAccess on license master.
- The Splunk IT Service Intelligence upgrade process is now complete. Objects disabled during the upgrade process are automatically reenabled. The ITSI shows the following message: IT Service Intelligence upgrade has completed successfully.
