topic Re: Re: Automatically mount most recent snapshot and connect to target. in Array Performance and Data Protection

Automatically Mount Most Recent Snapshot and Connect to Target

chris24 — Thu, 06 Feb 2014 11:26:20 GMT

HI All,

Hope it's useful to someone. Uses batch scripting, it's simple, working and tested.

Applications: Clone and mount most recent snapshot...

for development environment, scheduled it to mount the most recent snap at a DR environment so that devs would be working of the most recent data.
for backup
TODO - for backup and disable cache to prevent cache churn.
TODO - clone all recent snapshots for a volume collection and mount on ESX to simulate DR test without affecting replication.

or simply use it to quickly clone the most recent snapshot for a volume

Behaviour:,

BATCH - Finds most recent snapshot for the volume you specify, clones it (sets it online), fetches the new iqn which can then be passed into a powershell script to either connect the cloned snapshot volume to ESX or a Windows box. Uses text files to filter the results.

POWERSHELL - Connects only the new volume target IQN and sets any offline disk online. From testing it will retain any drive letter previously associated. !! Make sure you limit your volumes by intiator groups for security, as is best practice. !!

> can be run without executing anything to ensure you return the expected results.

Environment:

It's a batch script which executes a powershell script. Tested on MS2012 (will work on 2008 however the the ps1 will have to be changed) with array versions 1.4.x and 2.0.x. Run this script from the server you want to mount the volume onto.

Useful references:

For ps scripts using the iscsi cmdlets Windows PowerShell PS >_: New iSCSI cmdlets in PowerShell V3 Part - 1

Disclaimer:

Test it yourself use at your own peril ... etc etc. Read the comments.

Usage:

Create a password-less connection to the array by generating a SSH key between the server and array.

Install plink, puttyGEN and putty into c:\folder - PuTTY Download Page
Run puttyGen selecting 1024 bits, see screenshot
Save the key, selecting no to any passwords.
Copy the key exactly as shown below.

5. Connect to the array via putty using the login admin and your GUI password.

6. sshkey --add <name> --type rsa --key <paste the ssh key from previous step>

7. Open putty again prefix the array ip address with admin@ set the auth to SSH and point it at the key you saved in step 3, save the session with the name of your array.

> You can now run commands on the array via plink directly from the machine.

2. Open powershell as administrator run Set-ExecutionPolicy Unrestricted > Read what this means to you! Using the Set-ExecutionPolicy Cmdlet

3. Save the code below into c:\folder and rename i.e. recent-snapshot.bat save the powershell script as connect-target.ps1

4. Change the modifications in the script, add the session name of putty from step 1.7 above to the array-name=XXXXX

5. Execute - on first run comment out the sections where the snapshot is actually cloned to ensure you get the expected results.

name.bat VOLUME-NAME

i.e. C:\folder\recent-snapshot AA-SQL01-DB01

Batch file

:: This script will return the most recent snapshot for the volume and clone it, which sets it online.

:: Run the bat file followed by the name of the volume i.e. recent-snapshot.bat AA-GOLD-EXCH01-DB01

:: Once the volume has been created you can discover it on the host and connect to it.

:: This can be done automatically however if scheduled the volume target iqn changes, increases by 1 every time.

:: NUMBER OF MODIFICATIONS 5:

:: MOD 1. Having setup a SSH key with your array define the putty session name (array name) below

:: MOD 2. If you are mounting a snapshot for a backup and do not want it to churn your cache set to true NOT IN USE

@ echo off

set array-name=array-01

set full-backup=false

:: paramater taken from the command line - do not change

set volume=%1

echo Using the session %array-name%

echo .

:: MOD 3. If this is not a scheduled task you will want to comment the lines below to set the clone volume offline and delete it.

:: NOTE: there is a 20 second wait command, the ping, to allow the command to complete successfully

:: This is where you would invoke a powershell command if it is mounted in ESX | MS, on testing this is not neccessary for MS.

echo Removing the existing snapshot clone-%volume%

echo .

plink %array-name% vol --offline clone-%volume% --force

ping 192.0.2.2 -n 1 -w 20000 > nul

plink %array-name% vol --delete clone-%volume%

ping 192.0.2.2 -n 1 -w 20000 > nul

plink %array-name% snap --list > snap-list.txt

:: export the snapshot list

findstr /B /I %volume% c:snap-list.txt > snap.txt

:: find the snapshots related to the volume

for /f "tokens=2 delims= " %%a in (snap.txt) do (

:: retrieve the most recent volume collection name

set mostrecent=%%a

goto :break

)

:break

echo Mounting the snapshot %mostrecent% for volume %volume% it will be called clone-%volume% it will take a minute or so to appear within the GUI.

echo .

plink %array-name% vol --clone %volume% --snapname %mostrecent% --clonename clone-%volume%

:: MOD 4. If you don't need the IQN to pass onto Powershell to automatically mount the volume then comment out the section below.

ping 192.0.2.2 -n 1 -w 60000 > nul

:: wait for the volume to be cloned

plink %array-name% vol --info clone-%volume% > vol-info.txt

:: get the new volume information

findstr /B /I iSCSI c:vol-info.txt > iscsi.txt

:: find the right line

for /f "tokens=3 delims= " %%a in (iscsi.txt) do (

:: get the IQN of the new volume created this can be passed into a powershell script for esx or MS.

echo %%a

set iqn=%%a

)

:: MOD 5. from here you pass the parameter into a Vmware or MS powershell script. Script below will set the new volume online on MS 2012 only!.

:: The first time you mount the volume assign a drive letter and it will use the same letter every time, or you can add the command to the PS script.

:: NOTE: On testing we found you need to restart the SQL service, add it to the PS script.

powershell.exe -file c:\chris\connect-target.ps1 "%iqn%"

exit /b

Powershell to connect to target


param(

[string]$a

)

Write-Host $a

Get-IscsiSession | Register-IscsiSession

Connect-IscsiTarget -NodeAddress "$a"

Get-Disk | ?{$_.operationalstatus -eq "offline"} | Set-Disk -IsOffline 0

Get-Disk | ?{$_.operationalstatus -eq "online"}

NOTE: Initiator groups and ACL's are not replicated, if run at a DR site these will have to be defined.

Re: Automatically mount most recent snapshot and connect to target.

azagajewski27 — Fri, 21 Feb 2014 14:00:50 GMT

For those of us who are still using Windows 2008 R2 (probably most of us) and don't have PowerShell V4 which has the iSCSI commandlets, you can use iscsicli.exe to connect the LUNs and then diskpart.exe to put the new volumes online.

First, you need one more variable to account for the IP of the array, use the iSCSI discovery portal IP from the GUI:

set array-ip=10.0.1.50

Next, you have to attach and log in the new volume to the server using iscsicli.exe:

iscsicli.exe qaddtarget %iqn% %array-ip%

iscsicli.exe qlogintarget %iqn%

Then I like to wait 20 seconds for the O/S to recognize it and output all disks using diskpart.exe so that you can find the new volume which will be offline and, in turn, put it online (this won't work if you have multiple volumes offline, of course):

Create a new text file called listdisk.txt and save it in the same directory as all of your other batch files, executables, etc. All it needs to contain is this line:

list disk

Back to the batch file:

timeout /t 20 (much neater than pinging, btw)

diskpart.exe /s listdisk.txt > disks.txt

Create another new text file called offlinedisk.txt and save it in the same directory as all of your other batch files, executables, etc. All it needs to contain is these three lines:

select disk %disk%

online disk

attributes disk clear readonly

Continuing in the batch file, you need to parse the disks.txt for the offline disk and put it online:

findstr Offline disks.txt > offline.txt

for /f "tokens=2" %%b in (offline.txt) do set disk=%%b

diskpart.exe /s offlinedisk.txt

Hope this helps!

Re: Re: Automatically mount most recent snapshot and connect to target.

azagajewski27 — Fri, 21 Feb 2014 14:31:20 GMT

Two more items: First, a heartfelt thank you to Chris Aylott for putting together above code!

Second, I forgot to add a couple of statements that would remove the "Unrestricted" access to the new volume and then add the initiator group of the server to which you're connecting the new volume:

plink %array-name% vol --removeacl clone-%volume%

plink %array-name% vol --addacl clone-%volume% --initiatorgrp initiatorgroup

Re: Automatically mount most recent snapshot and connect to target.

azagajewski27 — Wed, 05 Mar 2014 15:44:02 GMT

OK, more updates:

I have found that because the serial of the cloned volume changes, Windows sees it as a new volume and does NOT always assign it the same drive letter or one at all. My backup software looks to backup specific drive letters on the backup media server therefore having the same one is critical. So I've added some more code that will take the new offline volume, find it and then assign it a drive letter. This also means that you have to pass another variable upon execution of the batch file to determine the drive letter. The new command will now be: C:\nimble\recent-snapshot.bat [Volume-Name] [Drive_Letter]

You also have to add a new line towards the beginning of the batch file to grab this drive letter:

:: drive letter to assign

set letter=%2

So, the new stuff needed to assign a drive letter to the new drive:

Add the following lines before the first iscsicli.exe line:

:: turn off automounting of basic volumes which will help later with drive letter assignment

diskpart.exe /s automountoff.txt

:: assign a drive letter to this new volume, make sure it's not already utilized but you have to get the volume from the previously offline disk first

diskpart.exe /s findnolettervol.txt > diskdetail.txt

:: find the no letter volume from the new disk and assign a letter to it

findstr Volume diskdetail.txt > nolettervol.txt

:: get the volume number

for /f "tokens=2" %%c in (nolettervol.txt) do set vol=%%c

:: assign a drive letter to this volume using the passed parameter when batch was launched

diskpart.exe /s assignlettervol.txt

:: turn back on automounting of volumes which helps with removable storage, etc

diskpart.exe /s automounton.txt

New text files to create, and their content:

automountoff.txt:

automount disable

automounton.txt:

automount enable

findnolettervol.txt:

select disk %disk%

detail disk

assignlettervol.txt:

select volume %vol%

attributes volume clear readonly

attributes volume clear hidden

assign letter %letter%

I've noticed that the volumes would come up as read only which is a problem because Windows needs to write a signature. Hence the added lines in the last text file.

Re: Automatically mount most recent snapshot and connect to target.

chris24 — Mon, 12 May 2014 17:26:10 GMT

Alex,

Great work!! I have added your contributions to the original script.

I have rolled up your diskpart scripts into one bat file so no need to create them by hand. Added an example for how I would call this when mounting daily snapshots to a SQL server, it also logs out of the target before deleting the snapshot ... found it kept the drive letter otherwise.

Re: Automatically mount most recent snapshot and connect to target.

chris24 — Mon, 12 May 2014 18:01:56 GMT

Unable to edit the original post --- continues from point 2.

2. Open powershell as administrator run Set-ExecutionPolicy Unrestricted > Read what this means to you! Using the Set-ExecutionPolicy Cmdlet

3. Save the code snippets below into c:\folder and rename accordingly i.e. recent-snapshot.bat.

4. Run the create-diskpart-scripts.bat once only.

5. Change the modifications in the script recent-snapshot.bat, add the session name of putty from step 1.7 above to the array-name=XXXXX

6. Execute - ensure the drive letters you want to use are free, the script will not disconnect any targets on the first run, I would recommend not having any connected so you can see it in action.

recent-snapshot VOLUME DRIVE-LETTER

i.e. C:\folder\recent-snapshot AA-SQL01-DB01 F

7. SQL-E-F-drive.bat is an example of usage with SQL

Batch file - create-diskpart-scripts.bat


echo list disk >> listdisk.txt

echo automount disable >> automountoff.txt

echo select disk %%disk%% >> findnolettervol.txt

echo detail disk >> findnolettervol.txt

echo select volume %%vol%% >> assignlettervol.txt

echo attributes volume clear readonly >> assignlettervol.txt

echo attributes volume clear hidden >> assignlettervol.txt

echo assign letter %%letter%% >> assignlettervol.txt

echo select disk %%disk%% >> offlinedisk.txt

echo online disk >> offlinedisk.txt

echo attributes disk clear readonly >> offlinedisk.txt

Batch file - recent-snapshot.bat

:: This script will:

:: 1. logout the existing iSCSI session

:: 2. delete any existing snapshot previously created by this script

:: 3. Create a clone copy of the most recent snapshot

:: 4. change the ACL

:: 5. connect the iSCSI target and associate it with a drive letter

:: Usage

:: Run the bat file followed by the name of the volume and the drive letter i.e. recent-snapshot.bat AA-GOLD-EXCH01-DB01 F

:: NUMBER OF MODIFICATIONS 3:

:: MOD 1. Having setup a SSH key with your array define the putty session name (array name) below

:: MOD 2. Array discovery IP, used for connecting the target

@ echo off

set array-name=array-connect

set discovery-ip=10.101.1.50

:: MOD 3. If you do not want the cloned snapshot to have the default permission of full access (0) define the init group.

set initator-grp=0

:: paramater taken from the command line - do not change

set volume=%1

set letter=%2

echo Using the session %array-name%

echo .

echo Disconnecting the iSCSI session for the existing mounted snpashot

echo .

:: Logout of the old target

:: Find the session ID of the existing connection - script has to be run once to get these values.

findstr "Session Id" %volume%login.txt > %volume%sessionID.txt

:: remove the gyberish leaving us with the SessionID

for /f "tokens=4 delims= " %%a in (%volume%sessionID.txt) do (

set sessionid=echo %%a

goto :break

)

:break

:: Logout the specified session ID

iscsicli logouttarget %sessionid%

echo Removing the existing snapshot clone-%volume%

echo .

plink %array-name% vol --offline Clone-%volume% --force

ping 192.0.2.2 -n 1 -w 20000 > nul

plink %array-name% vol --delete Clone-%volume%

ping 192.0.2.2 -n 1 -w 20000 > nul

:: Find the most recent snapshot

plink %array-name% snap --list > snap-list.txt

:: export the snapshot list

findstr /B /I %volume% c:snap-list.txt > snap.txt

:: find the snapshots related to the volume

for /f "tokens=2 delims= " %%a in (snap.txt) do (

:: retrieve the most recent volume collection name

set mostrecent=%%a

goto :break

)

:break

echo Mounting the snapshot %mostrecent% for volume %volume% it will be called Clone-%volume% it will take a minute or so to appear within the GUI.

echo .

plink %array-name% vol --clone %volume% --snapname %mostrecent% --clonename Clone-%volume%

:: wait

ping 192.0.2.2 -n 1 -w 20000 > nul

:: Set the ACL on the created clone

if %initiator-grp%==0 (GOTO :BREAK)

else (

plink %array-name% vol --removeacl Clone-%volume%

plink %array-name% vol --addacl Clone-%volume% --initiatorgrp %initiator-grp%)

:BREAK

:: Fetch the IQN

plink %array-name% vol --info Clone-%volume% > vol-info.txt

:: get the new volume information

findstr /B /I iSCSI c:vol-info.txt > iscsi.txt

:: find the right line

for /f "tokens=3 delims= " %%a in (iscsi.txt) do (

:: get the IQN of the new volume created this can be passed into a powershell script for esx or MS.

echo %%a > %volume%iqn.txt

set iqn=%%a

)

:: You now have the IQN - you can pass this into powershell via powershell.exe -file c:\putty\connect-target.ps1 "%iqn%"

:: Prevent the volume from being automatically added when the disk is onlined

diskpart.exe /s automountoff.txt

:: Connect the new target

iscsicli.exe qaddtarget %iqn% %discovery-ip%

:: login to the target and save the output to extract the sessionID required when dismounting the volume on next run

iscsicli.exe qlogintarget %iqn% > %volume%login.txt

ping 192.0.2.2 -n 1 -w 20000 > nul

:: fetch the disk

diskpart.exe /s listdisk.txt > disks.txt

:: online and clear read only attributes

findstr Offline disks.txt > offline.txt

for /f "tokens=2" %%b in (offline.txt) do set disk=%%b

diskpart.exe /s offlinedisk.txt

:: assign a drive letter to this new volume

diskpart.exe /s findnolettervol.txt > diskdetail.txt

:: find the no letter volume from the new disk and assign a letter to it

findstr Volume diskdetail.txt > nolettervol.txt

:: get the volume number

for /f "tokens=2" %%c in (nolettervol.txt) do set vol=%%c

:: online the volume with assigned drive letter

diskpart.exe /s assignlettervol.txt

exit /b

SQL server example - call this with a scheduled task

:: Stop SQL Service

net stop mssqlserver

call recent-snapshot.bat CA-SQL01-DB-F F

call recent-snapshot.bat CA-SQL01-LOG-E E

:: Start SQL Service

net start mssqlserver

EXIT

Re: Automatically Mount Most Recent Snapshot and Connect to Target

jliu79 — Tue, 21 Oct 2014 16:17:47 GMT

Hi Chris, I tried running your script but I'm getting error "GOTO was unexpected at this time" after "Mounting the snapshot %mostrecent% for volume %volume% it will be called Clone-%volume% it will take a minute or so to appear within the GUI. "

Any idea where could be wrong? Thanks.

P.S. I bypassed the ACL section now I'm having problem with this section:

for /f "tokens=3 delims= " %%a in (iscsi.txt) do (
:: get the IQN of the new volume created this can be passed into a powershell script for esx or MS.
echo %%a > %volume%iqn.txt
set iqn=%%a
)

The echo didn't give me the right iqn, it's just empty. The iscsi.txt does have the right info like this:

iSCSI target: iqn.2007-11.com.nimblestorage:clone-fileserver-v665c089a794a082d.000000a6.1f1600b7