How to disable FDE on MSA2060

[sdrake48]

MSA2060

What is process to turn off FDE once it has been enabled?  Is it as simple as deleting volumes and creating new volumes from the same SED drives?  Can it be disabled without losing all data? The documentation talks about locking and transporting.  I am looking to test performance with FDE on and FDE off for comparison.  I can reinstall OS on new volume if that is what is required, but would be nice to know if that can be avoided.  Since I turned on FDE, I created the password.  Can I use same password to decrypt this volume? If so, how, what is process?

[JonPaul · Accepted solution]

@sdrake48 
Starting with the important question:
Can FDE (Full Disk Encryption) be disabled without losing data?    No,  there is no way to "un-encrypt" the data on disk.
To disable FDE the disks and system need to be REPURPOSED.  Repurposing tells the SED (Self Encrypting Drive) to "spin" it's encryption key rendering ALL data on the drive permanently scrambled.

What is the process of disabling FDE?
  - Save your data
  - Remove all Volumes/Disk-Groups/Pool  (This is a double-check that you recognize that you are deleting the data from the system)
     - Easy way is to use the CLI>  delete pools A,B    {No confirmation in latest FW IN210R004 so make sure you recognize where you are and what you are doing}
  - Repurpose the system   {No confirmation but you have already deleted the Pools and data}
  - You can clear the FDE Lock Keys at any step or after or leave them but remember once the system is secured or re-secured it uses the lock key on the system if you don't remember the passphrase your data is vulnerable and you might end up doing this process again.

Performance differences using SED drives should be neglegible.  This is because the drive is always using it's encryption on the backend,  the FDE feature enabled/disabled is allowing the MSA controller access to the encryption with or without the passphrase -> Lock Key.  This also allows you to enable FDE at any time without a delay in the data being secured.  Solutions which use a controller based encryption require time to re-write the data in an encrypted form.

[sdrake48]

Hi,
Thanks. There is no data I care about. Only an installed OS that can be reinstalled at will. I am building/testing OS configurations for final configurations.

Prior to enabling FDE, I saw 2500MB/s from "hdparm -t /dev/sdb1". After enabling FDE, the same command yields 1200MB/sec. I tested with iozone (2GB filesize) and get the same result (1200MB/s read). I need to go back to FDE OFF, to verify the 2500MB/s number and test with iozone in that config.

I will delete Pools A and B, and hope to get back to state where FDE is not enabled. Then recreate my Pools, reinstall OS and test again.

[JonPaul]

@sdrake48 
1st please edit and remove your personal data   

I had not used hdparm before for performance.  I wonder how that works through a controller rather than direct to the drive. 
I am familiar with IOzone,  make sure your filesize is plenty big  (MSA controller Cache is 4GB,  should have 1GB of local WRITE cache)
The other caution I would make you aware of is if you are using SSDs they can have "interesting" performance until the drive has been conditioned.
What is your configuration?

[sdrake48]

2 x 900GB 15k SAS RAID1 Logical Drive 1 for OS

6 x 2.4TB 10k SAS RAID10 Logical Drive 2 for test

Both volumes in MSA2060 Pool A.  I have SSD's but no license yet so can't test with them.   SLES15 SP5

I can boot either a ML350 Gen11 or DL385 Gen11 from this OS (single initiator from each server, only one on at a time). SAS connect to e208e-p used as HBA.

hdparm -t read performance is the same from either ML350 or DL385. 

I tried 2 initiators from single e208e-p and noticed double SCSI devices, as well as hdparm read performance was half.  From this forum, was told of multipath.  Tried that, it worked.....but our Disaster Recovery software does not supoprt multipath (finds no disks) so can't use that.  So I am going with single initiator for each host. 

The final configuration is Master/Failover solution with 2 identical servers and external storage only. Only 1 server booted at a time.  In case of failure, master is turned off, failover is booted.  

Quickspecs say 8GB read cache per array (whatever an array is in MSA speak).   Using 2GB filesize with iozone for now.  Will increase size to see result. 

[sdrake48]

I have hdparm -t results back to HP ML370 G6 and SLES10.  All configs (G6 - G10) have a RAID controller, SAS druves, and RAID10 arrays.  Read results have always been very close to iozone or phoronix results (and far easier to obtain).  Mainly used for comparison, not absolute performance numbers.  Especially helpful if customer is creating is own vm's and you have no access to his storage but need to know what it is capable of.

[JonPaul]

@sdrake48 
Thanks for the configuration.  
Your configuration is not working as you are expecting.   If both disk-groups  RAID 1 and RAID10 are in the same pool then all data is being evenly divided between the 2 disk-groups.  This will result in "lumpy" performance, things get worse when the RAID 1 gets filled.   One 4MiB page of data will be read or written from the RAID 1 then the next read/written to the RAID 10.  There is no ability to PIN a volume to a specific set of disks in a Pool on an MSA 2060.   To acheive what you are desiring, you would have to create a Pool for your OS and a separate Pool for your data.
Here's a guide on more depth of how MSA Virtual storage works:  https://www.hpe.com/psnow/doc/a00103247enw
One other thing to keep in mind is if you will need to expand the system in the future.  ONLY the MSA-DP+ RAID level will allow incremental addition of drives.  All other RAID levels will require another disk-group and best practice would be to add the same configuration of disk-group.
A couple other sites to look at:
https://www.hpe.com/storage/MSAHealthCheck      - Free best practices checker
https://www.hpe.com/storage/MSAFirmware     - Direct links to the MSA firmware downloads

[sdrake48]

"There is no ability to PIN a volume to a specific set of disks in a Pool on an MSA 2060"   

I created Logical Drive 1 (volume) using 2 disks (the 15k ones). It asked for drives when I created it and I select those 2.  

I did the same for Logical Drive2 using the remaining 6 2.4TB 10k disks.  I disabled overcommit.

I performed unattended install of SLES15 SP5 with 2 logical drives.  The OS sees the right capacity for /dev/sda and /dev/sdb (matching the usable capacity of the two MSA Volumes.

When I run iozone on my 2nd logical drive (/dev/sdb), the led's on only the 6 2.4TB drives in Logical Drive 2 are active.  When I run iozone on the 1st logical drive, only the LED's on the first two drives are active.

Haven't I achieved a configuration where I have dedicated a MSA volume to a SLES scsi device?

[sdrake48]

By the way, only reason we are even considering MSA2060 in this configuration is there was no Gen11 update for the D3710 External Enclosure and the only Gen11 supported RAID controller for D3710 attachment (e208e-p) has no cache. 

[JonPaul]

@sdrake48 
Did the MSA default to creating 2 different Pools when you grouped the drives into RAID sets?
CLI>  show pools
CLI>  show disk-groups

The terminology is different between standard server Smart Arrays and the MSA so I may just be mis-understanding.
The E208e-p acts as a passthru and there is nothing to configure (or available to configure) in the Smart Storage Administrator

[sdrake48]

Yes, e208 is acting as HBA, no raid configuration.

MSA SMU has 2 disk groups, both in Pool A.  Nothing in Pool B.

[sdrake48]

This worked.  FDE is now off.  Just had to recreate everything and reinstall OS.   Strangely enough, hdparm -t on 2 disk volume is 1/2 the performance with FDE off.  Not the case on 6 disk volume where hdparm -t performance was close to identical with FDE on.   But from your comments on how the MSA handles volumes it seems its not possible to know where the reads are physically happening.  

That's a whole different problem: How to maximize performance of MSA2060 when used for a single OS (SLES15 in my case) installation. No esxi, no vmware, just native SLES15 installation (boot from SAN).  How to configure a MSA to replicate the I/O throughput obtained with multiple logical drives and multiple RAID controllers on a server with internal storage. 

[sdrake48]

All logical drives (arrays) are in a single pool.  I create an array in Pool A, then immediately provision that volume (selecting 'use all space'). Then I create another array in Pool A and provision it (use all space).  Then I do it again.  I installed SLES15 SP5 from unattended install xml files that configures 3 logical drives.  I have to fiddle with kernel boot parameters so that /dev/sda is always /dev/sda on every boot (disable asynchronous probing). This has been installed and running for over a month...no data has migrated between volumes. The single OS that is installed using these 3 volumes is still reporting the exact same sizes (df -h) for each logical drive.  By all appearances, I have dedicated each of 3 volumes to specific hard drives.

The original reason for this case has been resolved.  To disable FDE you must first remove all volumes, disk groups, etc.   All data is lost.  Then you can disable FDE, recreate disk groups, volumes, etc.