[This is a repost of an original article from May 2020 on blog.purestorage.com]
Over the years, I have been involved in many, many data migration projects. Believe me, I never want to go through those overnight and weekend marathons again, with all the associated application outages, rollbacks, complexity, and more.
If you had told me back then there would be a simple, push-button way to migrate from one storage array to another, while the application was still running, and there was little or no impact to the application users, I would have told you that you were insane. Well, fast-forward to today: migration nirvana is within your grasp if you are migrating from a one Pure Storage FlashArray to another when you use the FlashArray Ansible Collections.
The rest of this blog post will show how to use Ansible to perform a volume migration from one FlashArray to another, with live IO being performed to the filesystem by the underlying volume. This allows applications to remain active during the whole migration process with little or no impact on their users, and depending on the application there may be a slight increase in response times.
The process has been recorded in this short video, and the playbooks that are used in the video are detailed later in this blog post.
Notice the timestamps in the video. There are no smoke and mirrors here. This is all real-time.
The video shows the creation of a simple volume on a host connected to a single FlashArray running an application that is writing data to the volume. This volume is then migrated, whilst the application is still running, to another FlashArray.
The steps to perform the migration are:
- Create an ActiveCluster between the two arrays
- Create an ActiveCluster pod
- Move the volume to the pod
- Stretch the pod and let the data synchronise
- Move the volume out of the pod onto the second array
- Break the ActiveCluster connection
- Cleanup the first array.
As I stated above, the first part of this video creates a simple volume, then formats it and mounts it to be ready for an application to write data. The playbook to do this is:
- name: Create volume for migration example
hosts: localhost
gather_facts: true
vars_files:
- mig-vars.yaml
tasks:
- name: Get source FlashArray info
purestorage.flasharray.purefa_info:
gather_subset:
- minimum
- network
- interfaces
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
register: src_array_info
- name: Create volume for migration
purestorage.flasharray.purefa_volume:
name: "{{ migration_volume }}"
size: 10G
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Get serial number of migration volume
purestorage.flasharray.purefa_info:
gather_subset: volumes
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
register: volumes_data
- set_fact:
volume_serial: "{{ volumes_data.purefa_info.volumes[migration_volume].serial }}"
- name: Create host object on source array and connect volume
purestorage.flasharray.purefa_host:
host: "{{ ansible_hostname }}"
iqn: "{{ ansible_iscsi_iqn }}"
volume: "{{ migration_volume }}"
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Discover source FlashArray for iSCSI
open_iscsi:
show_nodes: yes
discover: yes
portal: "{{ src_array_info.purefa_info.network[src_iscsi_port].address }}"
register: src_iscsi_iqn
- name: Connect to source FlashArray over iSCSI
open_iscsi:
target: "{{ src_iscsi_iqn.nodes[0] }}"
login: yes
- name: Force multipath rescan
command: /usr/sbin/multipath -r
- name: Get multipath device for migration volume
shell:
cmd: /usr/sbin/multipath -ll | grep -i {{ volume_serial }} | awk '{print $2}'
register: mpath_dev
- name: Format migration volume
filesystem:
fstype: ext4
dev: '/dev/{{ mpath_dev.stdout }}'
- name: Mount migration volume
mount:
path: "{{ mount_path }}"
fstype: ext4
src: '/dev/{{ mpath_dev.stdout }}'
state: mounted
You can also find this in the Pure Storage GitHub account.
The application I use to write data to the volume is a very simple random text generator, which ensures data is continually being written to the volume during the live migration.
The main Ansible playbook performs the actions listed earlier:
- First, start by creating an ActiveCluster link between the source and target FlashArrays.
![Ansible Content Collection Migration](https://blog.purestorage.com/wp-content/uploads/2020/03/migration-ac-link-1.gif)
- Next, create a pod and move the volume into the pod.
![Create ActiveCluster Pod for migration](https://blog.purestorage.com/wp-content/uploads/2020/03/migration-create-pod-1.gif)
- Stretch the pod to the target array, which allows the pod to synchronize the volume across the two FlashArrays.
![Stretched ActiveCluster Pod for Ansible Migration](https://blog.purestorage.com/wp-content/uploads/2020/03/migration-stretch-1.gif)
At this point, the playbook uses a loop process to wait for the pod to fully synchronize. This is the core part of the migration, and you can see that data is still being written to the volume all the way. Imagine your production database being migrated with zero downtime.
- After the volume has synchronized, it is connected to the second host and the host sees connections for the volume from both arrays.
![](https://blog.purestorage.com/wp-content/uploads/2020/03/migration-2-hosts-1.gif)
- The volume is disconnected from the source array in the active cluster pod construct. The pod is unstretched and the volume is removed from the pod.
![](https://blog.purestorage.com/wp-content/uploads/2020/03/migration-unstretch-2.gif)
- Finally, all references to the ActiveCluster are removed, and the source array connections to the host are removed.
The playbook used to perform this migration is:
- name: Live migration using ActiveCluster
hosts: localhost
gather_facts: true
vars_files:
- mig-vars.yaml
tasks:
- name: Get source FlashArray info
purestorage.flasharray.purefa_info:
gather_subset:
- minimum
- network
- interfaces
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
register: src_array_info
- name: Get destination FlashArray info
purestorage.flasharray.purefa_info:
gather_subset:
- minimum
- network
- interfaces
fa_url: "{{ dst_array_ip }}"
api_token: "{{ dst_array_api }}"
register: dst_array_info
- name: Connect arrays in ActiveCluster configuration
purestorage.flasharray.purefa_connect:
target_url: "{{ dst_array_ip }}"
target_api: "{{ dst_array_api }}"
connection: sync
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Create migration pod
purestorage.flasharray.purefa_pod:
name: "{{ migration_pod }}"
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Move migration volume to migration pod
purestorage.flasharray.purefa_volume:
name: "{{ migration_volume }}"
move: "{{ migration_pod }}"
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Stretch migration pod to destination array
purestorage.flasharray.purefa_pod:
name: "{{ migration_pod }}"
stretch: "{{ dst_array_info['purefa_info']['default']['array_name'] }}"
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Wait for pod sync
purestorage.flasharray.purefa_info:
gather_subset: pods
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
register: output
retries: 40
delay: 5
until: "output | json_query('purefa_info.pods.\"{{ migration_pod }}\".arrays[].status') == ['online', 'online']"
- name: Create host object on destination array
purestorage.flasharray.purefa_host:
host: "{{ ansible_hostname }}"
iqn: "{{ ansible_iscsi_iqn }}"
volume: "{{ migration_pod }}::{{ migration_volume }}"
fa_url: "{{ dst_array_ip }}"
api_token: "{{ dst_array_api }}"
- name: Discover destination FlashArray using iSCSI
open_iscsi:
show_nodes: yes
discover: yes
portal: "{{ dst_array_info.purefa_info.network[dst_iscsi_port].address }}"
register: dst_iscsi_iqn
- name: Connect to destination FlashArray over iSCSI
open_iscsi:
target: "{{ dst_iscsi_iqn.nodes[0] }}"
login: yes
- name: Ensure new multipath links from destination array are connected
command: /usr/sbin/multipath -r
- debug:
msg: "Volume fully sync'ed and ready for removal from source array"
- pause:
- name: Disconnect migration volume from host on source array
purestorage.flasharray.purefa_host:
volume: "{{ migration_pod }}::{{ migration_volume }}"
host: "{{ ansible_hostname }}"
state: absent
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Unstretch pod from source array
purestorage.flasharray.purefa_pod:
name: "{{ migration_pod }}"
state: absent
stretch: "{{ src_array_info['purefa_info']['default']['array_name'] }}"
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Move migrated volume out of pod on destination array
purestorage.flasharray.purefa_volume:
name: '{{ migration_pod }}::{{ migration_volume }}'
move: local
fa_url: "{{ dst_array_ip }}"
api_token: "{{ dst_array_api }}"
- name: Remove old multipath links to source array
command: /usr/sbin/multipath -r
- debug:
msg: "Volume fully migrated to destination array. Ready to clean up both arrays."
- pause:
- name: Eradicate migration pod on destination array
purestorage.flasharray.purefa_pod:
name: "{{ migration_pod }}"
state: absent
eradicate: true
fa_url: "{{ dst_array_ip }}"
api_token: "{{ dst_array_api }}"
- name: Cleanup hanging migration pod on source array
purestorage.flasharray.purefa_pod:
name: "{{ migration_pod }}.restretch"
state: absent
eradicate: true
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Disconnect arrays from ActiveCluster mode
purestorage.flasharray.purefa_connect:
target_url: "{{ dst_array_ip }}"
target_api: "{{ dst_array_api }}"
state: absent
fa_url: "{{ src_array_ip }}"
api_token: "{{ src_array_api }}"
- name: Remove old multipath links to source array
command: /usr/sbin/multipath -r
- name: Get source array IQN
shell:
cmd: /usr/sbin/iscsiadm -m node | grep {{ src_array_info.purefa_info.network[src_iscsi_port].address }} | awk '{print $2}'
register: src_iqn
- name: Logout iSCSI sessions to source array
shell:
cmd: /usr/sbin/iscsiadm -m node -T {{ src_iqn.stdout }} -u
- name: Delete iSCSI sessions to source array
shell:
cmd: /usr/sbin/iscsiadm -m node -o delete -T {{ src_iqn.stdout }}
You can also download from GitHub.
You wouldn’t do this next step in a production environment, but the video continues and uses a playbook to clean up the demonstration environment. This unmounts the file system from the host, deletes the volume on the target array, and removes all references to the target array from the host. The playbook used for this is:
- name: Cleanup environment after migration complete
hosts: localhost
gather_facts: true
vars_files:
- mig-vars.yaml
tasks:
- name: Get destination array information
purestorage.flasharray.purefa_info:
gather_subset:
- network
- volumes
fa_url: "{{ dst_array_ip }}"
api_token: "{{ dst_array_api }}"
register: dst_array_info
- set_fact:
volume_serial: "{{ dst_array_info.purefa_info.volumes[migration_volume].serial }}"
- name: Unmount filesystem
mount:
path: "{{ mount_path }}"
state: absent
- name: Get multipath device of migrated volume
shell:
cmd: /usr/sbin/multipath -ll | grep -i {{ volume_serial }} | awk '{print $2}'
register: mpath_dev
- name: Delete host object from destination array
purestorage.flasharray.purefa_host:
host: "{{ ansible_hostname }}"
state: absent
fa_url: "{{ dst_array_ip }}"
api_token: "{{ dst_array_api }}"
- name: Delete migrated volume from destination array
purestorage.flasharray.purefa_volume:
name: "{{ migration_volume }}"
state: absent
eradicate: true
fa_url: "{{ dst_array_ip }}"
api_token: "{{ dst_array_api }}"
- name: Remove multipath device
shell:
cmd: /usr/sbin/multipath -r /dev/{{ mpath_dev.stdout }}
- name: Flush multipath cache
shell:
cmd: /usr/sbin/multipath -F
- name: Get destination array IQN
shell:
cmd: /usr/sbin/iscsiadm -m node | grep {{ dst_array_info.purefa_info.network[dst_iscsi_port].address }} | awk '{print $2}'
register: dst_iqn
- name: Logout iSCSI sessions to destination array
shell:
cmd: /usr/sbin/iscsiadm -m node -T {{ dst_iqn.stdout }} -u
- name: Delete iSCSI sessions to destination array
shell:
cmd: /usr/sbin/iscsiadm -m node -o delete -T {{ dst_iqn.stdout }}
This is also available on the Pure Storage GitHub account.
Each of these playbooks calls a variable file that holds environmental information that allows the migration to occur. An example variable file is available.
If you download these playbooks, please ensure you update the variables file with your settings.
This example only uses iSCSI for the host-volume connection, but I’m sure you could adapt it to support Fibre Channel connectivity. If you do, please upload a copy to the Pure Storage Ansible playbooks examples repository