Automatically Mount Most Recent Snapshot and Connect to Target

[chris24]

HI All,

Hope it's useful to someone. Uses batch scripting, it's simple, working and tested.

Applications: Clone and mount most recent snapshot...

1. for development environment, scheduled it to mount the most recent snap at a DR environment so that devs would be working of the most recent data.
2. for backup
3. TODO - for backup and disable cache to prevent cache churn.
4. TODO - clone all recent snapshots for a volume collection and mount on ESX to simulate DR test without affecting replication.

or simply use it to quickly clone the most recent snapshot for a volume

Behaviour:,

BATCH - Finds most recent snapshot for the volume you specify, clones it (sets it online), fetches the new iqn which can then be passed into a powershell script to either connect the cloned snapshot volume to ESX or a Windows box. Uses text files to filter the results.

POWERSHELL - Connects only the new volume target IQN and sets any offline disk online. From testing it will retain any drive letter previously associated. !! Make sure you limit your volumes by intiator groups for security, as is best practice. !!

> can be run without executing anything to ensure you return the expected results.

Environment:

It's a batch script which executes a powershell script. Tested on MS2012 (will work on 2008 however the the ps1 will have to be changed) with array versions 1.4.x and 2.0.x. Run this script from the server you want to mount the volume onto.

Useful references:

For ps scripts using the iscsi cmdlets Windows PowerShell  PS >_: New iSCSI cmdlets in PowerShell V3 Part - 1

Disclaimer:

Test it yourself use at your own peril ... etc etc. Read the comments.

Usage:

1. Create a password-less connection to the array by generating a SSH key between the server and array.

1. Install plink, puttyGEN and putty into c:\folder - PuTTY Download Page
2. Run puttyGen selecting 1024 bits, see screenshot
3. Save the key, selecting no to any passwords.
4. Copy the key exactly as shown below.

     5. Connect to the array via putty using the login admin and your GUI password.

     6. sshkey --add <name> --type rsa --key <paste the ssh key from previous step>

     7. Open putty again prefix the array ip address with admin@ set the auth to SSH and point it at the key you saved in step 3, save the session with the name of your array.

> You can now run commands on the array via plink directly from the machine.

2. Open powershell as administrator run Set-ExecutionPolicy Unrestricted > Read what this means to you! Using the Set-ExecutionPolicy Cmdlet

3. Save the code below into c:\folder and rename i.e. recent-snapshot.bat save the powershell script as connect-target.ps1

4. Change the modifications in the script, add the session name of putty from step 1.7 above to the array-name=XXXXX

5. Execute - on first run comment out the sections where the snapshot is actually cloned to ensure you get the expected results.

    name.bat VOLUME-NAME

    i.e. C:\folder\recent-snapshot AA-SQL01-DB01

Batch file

:: This script will return the most recent snapshot for the volume and clone it, which sets it online.
:: Run the bat file followed by the name of the volume i.e. recent-snapshot.bat AA-GOLD-EXCH01-DB01
:: Once the volume has been created you can discover it on the host and connect to it.
:: This can be done automatically however if scheduled the volume target iqn changes, increases by 1 every time.


:: NUMBER OF MODIFICATIONS 5:
:: MOD 1. Having setup a SSH key with your array define the putty session name (array name) below
:: MOD 2. If you are mounting a snapshot for a backup and do not want it to churn your cache set to true NOT IN USE
@ echo off
set array-name=array-01
set full-backup=false
:: paramater taken from the command line - do not change
set volume=%1


  echo Using the session %array-name%
  echo .
  echo .


:: MOD 3. If this is not a scheduled task you will want to comment the lines below to set the clone volume offline and delete it.
:: NOTE: there is a 20 second wait command, the ping, to allow the command to complete successfully
:: This is where you would invoke a powershell command if it is mounted in ESX | MS, on testing this is not neccessary for MS.
echo Removing the existing snapshot clone-%volume%
  echo .
  echo .
plink %array-name% vol --offline clone-%volume% --force
ping 192.0.2.2 -n 1 -w 20000 > nul
plink %array-name% vol --delete clone-%volume%
ping 192.0.2.2 -n 1 -w 20000 > nul


plink %array-name% snap --list > snap-list.txt
:: export the snapshot list
findstr /B /I %volume% c:snap-list.txt > snap.txt
:: find the snapshots related to the volume


for /f "tokens=2 delims= " %%a in (snap.txt) do (
:: retrieve the most recent volume collection name
  set mostrecent=%%a
  goto :break
  )
  :break
  echo Mounting the snapshot %mostrecent% for volume %volume% it will be called clone-%volume% it will take a minute or so to appear within the GUI.
  echo .
  plink %array-name% vol --clone %volume% --snapname %mostrecent% --clonename clone-%volume%


:: MOD 4. If you don't need the IQN to pass onto Powershell to automatically mount the volume then comment out the section below.
ping 192.0.2.2 -n 1 -w 60000 > nul
:: wait for the volume to be cloned
plink %array-name% vol --info clone-%volume% > vol-info.txt
:: get the new volume information
findstr /B /I iSCSI  c:vol-info.txt > iscsi.txt
:: find the right line
for /f "tokens=3 delims= " %%a in (iscsi.txt) do (
:: get the IQN of the new volume created this can be passed into a powershell script for esx or MS.
  echo %%a
  set iqn=%%a
  )
:: MOD 5. from here you pass the parameter into a Vmware or MS powershell script. Script below will set the new volume online on MS 2012 only!.
:: The first time you mount the volume assign a drive letter and it will use the same letter every time, or you can add the command to the PS script.
:: NOTE: On testing we found you need to restart the SQL service, add it to the PS script.


powershell.exe -file c:\chris\connect-target.ps1 "%iqn%"

exit /b

Powershell to connect to target

param(
[string]$a
)
Write-Host $a
Get-IscsiSession | Register-IscsiSession
Connect-IscsiTarget -NodeAddress "$a"
Get-Disk | ?{$_.operationalstatus -eq "offline"} | Set-Disk -IsOffline 0
Get-Disk | ?{$_.operationalstatus -eq "online"}

NOTE: Initiator groups and ACL's are not replicated, if run at a DR site these will have to be defined.

[azagajewski27]

For those of us who are still using Windows 2008 R2 (probably most of us) and don't have PowerShell V4 which has the iSCSI commandlets, you can use iscsicli.exe to connect the LUNs and then diskpart.exe to put the new volumes online.

First, you need one more variable to account for the IP of the array, use the iSCSI discovery portal IP from the GUI:

set array-ip=10.0.1.50

Next, you have to attach and log in the new volume to the server using iscsicli.exe:

iscsicli.exe qaddtarget %iqn% %array-ip%

iscsicli.exe qlogintarget %iqn%

Then I like to wait 20 seconds for the O/S to recognize it and output all disks using diskpart.exe so that you can find the new volume which will be offline and, in turn, put it online (this won't work if you have multiple volumes offline, of course):

Create a new text file called listdisk.txt and save it in the same directory as all of your other batch files, executables, etc.  All it needs to contain is this line:

list disk

Back to the batch file:

timeout /t 20 (much neater than pinging, btw)

diskpart.exe /s listdisk.txt > disks.txt

Create another new text file called offlinedisk.txt and save it in the same directory as all of your other batch files, executables, etc.  All it needs to contain is these three lines:

select disk %disk%

online disk

attributes disk clear readonly

Continuing in the batch file, you need to parse the disks.txt for the offline disk and put it online:

findstr Offline disks.txt > offline.txt

for /f "tokens=2" %%b in (offline.txt) do set disk=%%b

diskpart.exe /s offlinedisk.txt

Hope this helps!

[azagajewski27]

Two more items:  First, a heartfelt thank you to Chris Aylott for putting together above code!

Second, I forgot to add a couple of statements that would remove the "Unrestricted" access to the new volume and then add the initiator group of the server to which you're connecting the new volume:

plink %array-name% vol --removeacl clone-%volume%

plink %array-name% vol --addacl clone-%volume% --initiatorgrp initiatorgroup

[azagajewski27]

OK, more updates:

I have found that because the serial of the cloned volume changes, Windows sees it as a new volume and does NOT always assign it the same drive letter or one at all.  My backup software looks to backup specific drive letters on the backup media server therefore having the same one is critical.  So I've added some more code that will take the new offline volume, find it and then assign it a drive letter.  This also means that you have to pass another variable upon execution of the batch file to determine the drive letter.  The new command will now be: C:\nimble\recent-snapshot.bat [Volume-Name] [Drive_Letter]

You also have to add a new line towards the beginning of the batch file to grab this drive letter:

:: drive letter to assign

set letter=%2

So, the new stuff needed to assign a drive letter to the new drive:

Add the following lines before the first iscsicli.exe line:

:: turn off automounting of basic volumes which will help later with drive letter assignment

diskpart.exe /s automountoff.txt

:: assign a drive letter to this new volume, make sure it's not already utilized but you have to get the volume from the previously offline disk first

diskpart.exe /s findnolettervol.txt > diskdetail.txt

:: find the no letter volume from the new disk and assign a letter to it

findstr Volume diskdetail.txt > nolettervol.txt

:: get the volume number

for /f "tokens=2" %%c in (nolettervol.txt) do set vol=%%c

:: assign a drive letter to this volume using the passed parameter when batch was launched

diskpart.exe /s assignlettervol.txt

:: turn back on automounting of volumes which helps with removable storage, etc

diskpart.exe /s automounton.txt

New text files to create, and their content:

automountoff.txt:

     automount disable

automounton.txt:

     automount enable

findnolettervol.txt:

     select disk %disk%

     detail disk

assignlettervol.txt:

     select volume %vol%

     attributes volume clear readonly

     attributes volume clear hidden

     assign letter %letter%

I've noticed that the volumes would come up as read only which is a problem because Windows needs to write a signature.  Hence the added lines in the last text file.

[chris24]

Alex,

Great work!! I have added your contributions to the original script.

I have rolled up your diskpart scripts into one bat file so no need to create them by hand. Added an example for how I would call this when mounting daily snapshots to a SQL server, it also logs out of the target before deleting the snapshot ... found it kept the drive letter otherwise.

[chris24]

Unable to edit the original post --- continues from point 2.

2. Open powershell as administrator run Set-ExecutionPolicy Unrestricted > Read what this means to you! Using the Set-ExecutionPolicy Cmdlet

3. Save the code snippets below into c:\folder and rename accordingly i.e. recent-snapshot.bat.

4. Run the create-diskpart-scripts.bat once only.

5. Change the modifications in the script recent-snapshot.bat, add the session name of putty from step 1.7 above to the array-name=XXXXX

6. Execute - ensure the drive letters you want to use are free, the script will not disconnect any targets on the first run, I would recommend not having any connected so you can see it in action.

     recent-snapshot VOLUME DRIVE-LETTER

    i.e. C:\folder\recent-snapshot AA-SQL01-DB01 F

7. SQL-E-F-drive.bat is an example of usage with SQL

Batch file - create-diskpart-scripts.bat

echo list disk >> listdisk.txt
echo automount disable >> automountoff.txt
echo select disk %%disk%% >> findnolettervol.txt
echo detail disk >> findnolettervol.txt
echo select volume %%vol%% >> assignlettervol.txt
echo attributes volume clear readonly >> assignlettervol.txt
echo attributes volume clear hidden >> assignlettervol.txt
echo assign letter %%letter%% >> assignlettervol.txt
echo select disk %%disk%% >> offlinedisk.txt
echo online disk >> offlinedisk.txt
echo attributes disk clear readonly >> offlinedisk.txt

Batch file - recent-snapshot.bat

:: This script will:
:: 1. logout the existing iSCSI session
:: 2. delete any existing snapshot previously created by this script
:: 3. Create a clone copy of the most recent snapshot
:: 4. change the ACL
:: 5. connect the iSCSI target and associate it with a drive letter

:: Usage
:: Run the bat file followed by the name of the volume and the drive letter i.e. recent-snapshot.bat AA-GOLD-EXCH01-DB01 F

:: NUMBER OF MODIFICATIONS 3:
:: MOD 1. Having setup a SSH key with your array define the putty session name (array name) below
:: MOD 2. Array discovery IP, used for connecting the target
@ echo off
set array-name=array-connect
set discovery-ip=10.101.1.50

:: MOD 3. If you do not want the cloned snapshot to have the default permission of full access (0) define the init group.
set initator-grp=0

:: paramater taken from the command line - do not change
set volume=%1
set letter=%2

  echo Using the session %array-name%
  echo .
  echo .

echo Disconnecting the iSCSI session for the existing mounted snpashot
  echo .
  echo .

:: Logout of the old target
:: Find the session ID of the existing connection - script has to be run once to get these values.
findstr "Session Id" %volume%login.txt > %volume%sessionID.txt
:: remove the gyberish leaving us with the SessionID
for /f "tokens=4 delims= " %%a in (%volume%sessionID.txt) do (
  set sessionid=echo %%a
  goto :break
  )
  :break

:: Logout the specified session ID
iscsicli logouttarget %sessionid%

echo Removing the existing snapshot clone-%volume%
  echo .
  echo .
plink %array-name% vol --offline Clone-%volume% --force
ping 192.0.2.2 -n 1 -w 20000 > nul
plink %array-name% vol --delete Clone-%volume%
ping 192.0.2.2 -n 1 -w 20000 > nul

:: Find the most recent snapshot

plink %array-name% snap --list > snap-list.txt
:: export the snapshot list
findstr /B /I %volume% c:snap-list.txt > snap.txt
:: find the snapshots related to the volume

for /f "tokens=2 delims= " %%a in (snap.txt) do (
:: retrieve the most recent volume collection name
  set mostrecent=%%a
  goto :break
  )
  :break
  echo Mounting the snapshot %mostrecent% for volume %volume% it will be called Clone-%volume% it will take a minute or so to appear within the GUI.
  echo .
plink %array-name% vol --clone %volume% --snapname %mostrecent% --clonename Clone-%volume%

:: wait
ping 192.0.2.2 -n 1 -w 20000 > nul

:: Set the ACL on the created clone
if %initiator-grp%==0 (GOTO :BREAK)
  else (
plink %array-name% vol --removeacl Clone-%volume%
plink %array-name% vol --addacl Clone-%volume% --initiatorgrp %initiator-grp%)
:BREAK

:: Fetch the IQN
plink %array-name% vol --info Clone-%volume% > vol-info.txt
:: get the new volume information
findstr /B /I iSCSI  c:vol-info.txt > iscsi.txt
:: find the right line
for /f "tokens=3 delims= " %%a in (iscsi.txt) do (
:: get the IQN of the new volume created this can be passed into a powershell script for esx or MS.
  echo %%a > %volume%iqn.txt
  set iqn=%%a
  )

:: You now have the IQN - you can pass this into powershell via powershell.exe -file c:\putty\connect-target.ps1 "%iqn%"

:: Prevent the volume from being automatically added when the disk is onlined
diskpart.exe /s automountoff.txt

:: Connect the new target
iscsicli.exe qaddtarget %iqn% %discovery-ip%
:: login to the target and save the output to extract the sessionID required when dismounting the volume on next run
iscsicli.exe qlogintarget %iqn% > %volume%login.txt
ping 192.0.2.2 -n 1 -w 20000 > nul

:: fetch the disk
diskpart.exe /s listdisk.txt > disks.txt

:: online and clear read only attributes
findstr Offline disks.txt > offline.txt
for /f "tokens=2" %%b in (offline.txt) do set disk=%%b
diskpart.exe /s offlinedisk.txt

:: assign a drive letter to this new volume
diskpart.exe /s findnolettervol.txt > diskdetail.txt
:: find the no letter volume from the new disk and assign a letter to it
findstr Volume diskdetail.txt > nolettervol.txt
:: get the volume number
for /f "tokens=2" %%c in (nolettervol.txt) do set vol=%%c
:: online the volume with assigned drive letter
diskpart.exe /s assignlettervol.txt

  exit /b

SQL server example - call this with a scheduled task

:: Stop SQL Service
net stop mssqlserver

call recent-snapshot.bat CA-SQL01-DB-F F
call recent-snapshot.bat CA-SQL01-LOG-E E

:: Start SQL Service
net start mssqlserver

EXIT

[jliu79]

Hi Chris, I tried running your script but I'm getting error "GOTO was unexpected at this time" after "Mounting the snapshot %mostrecent% for volume %volume% it will be called Clone-%volume% it will take a minute or so to appear within the GUI. "

Any idea where could be wrong? Thanks.

P.S. I bypassed the ACL section now I'm having problem with this section:

1. for /f "tokens=3 delims= " %%a in (iscsi.txt) do ( 
2. :: get the IQN of the new volume created this can be passed into a powershell script for esx or MS. 
3.   echo %%a > %volume%iqn.txt 
4.   set iqn=%%a 
5.   ) 

The echo didn't give me the right iqn, it's just empty. The iscsi.txt does have the right info like this:

iSCSI target: iqn.2007-11.com.nimblestorage:clone-fileserver-v665c089a794a082d.000000a6.1f1600b7