Tuesday, October 1, 2019

Configuring VMware vSphere vVol DataStore - NetApp ONTAP 9.6 & VSC 9.6

What is Virtual Volumes (vVols)?

Eliminate physical containers and enable more flexible consumption of resources with Virtual Volumes (vVols). This integration and management framework for external storage provides finer control at the VM-level, streamlines storage operations, and offers flexibility of choice.

vVols is an integration and management framework that virtualizes SAN/NAS arrays, enabling a more efficient operational model that is optimized for virtualized environments and centered on the application instead of the infrastructure. vVols simplifies operations through policy-driven automation that enables more agile storage consumption for virtual machines and dynamic adjustments in real time, when they are needed. It simplifies the delivery of storage service levels to individual applications by providing finer control of hardware resources and native array-based data services that can be instantiated with virtual machine granularity.

vVols are VMDK granular storage entities exported by storage arrays. vVols are exported to the ESXi host through a small set of protocol end-points (PE). Protocol Endpoints are part of the physical storage fabric, and they establish a data path from virtual machines to their respective vVols on demand. Storage systems enable data services on vVols. The results of these data services are newer vVols. Data services, configuration and management of virtual volume systems is exclusively done out-of-band with respect to the data path. vVols can be grouped into logical entities called storage containers (SC) for management purposes. The existence of storage containers is limited to the out-of-band management channel.

vVols and Storage Containers (SC) form the virtual storage fabric. Protocol Endpoints (PE) are part of the physical storage fabric.

By using a special set of APIs called vSphere APIs for Storage Awareness (VASA), the storage system becomes aware of the vVols and their associations with the relevant virtual machines.

Characteristics of vVols:

· No File System.

· ESX manages the array through VASA (vSphere APIs for Storage Awareness) APIs.

· Arrays are logically partitioned into containers, called Storage Containers.

· Virtual machine disks, called vVols, stored natively on the Storage Containers.

· IO from ESXi host to the storage array is addressed through an access point called, Protocol Endpoint (PE).

· Data Services are offloaded to the array. Snapshot, Replication, Encryption.

· Managed through storage policy-based management (SPBM) framework.

VASA (vSphere APIs for Storage Awareness) APIs (VP)

A vVols storage provider, also called a VASA provider is a software component that acts as a storage awareness service for vSphere. The provider mediates out-of-band communication between the vCenter Server and ESXi hosts on one side and a storage system on the other.

· Software component developed by Storage Array Vendors.

· ESX and vCenter Server connect to VASA Provider.

· Provides Storage awareness services.

· Single VASA Provider can manages multiple arrays.

· Supports VASA APIs exported by the ESXi host.

· VASA Provider can be implemented within the array’s management server or firmware.

· Responsible for creating vVols.

Storage Containers (SC)

Unlike traditional LUN and NFS based vSphere storage, the vVols functionality does not require pre-configured volumes on a storage side. Instead, vVols uses a storage container, which is a pool of raw storage capacity or an aggregation of storage capabilities that a storage system can provide to vVols.

· Logical storage constructs for grouping of vVols.

· Logically partition or isolate virtual machines with diverse storage needs and requirement.

· A single Storage Container can be simultaneously accessed via multiple Protocol Endpoints.

· Desired capabilities are applied to the Storage Containers.

· VASA Provider discovers Storage Container and reports to the vCenter Server.

· Any new virtual machines that are created are subsequently provisioned in the Storage Container.

Protocol Endpoint (PE)

Although storage systems manage all aspects of vVols, ESXi hosts have no direct access to vVols on the storage side. Instead, ESXi hosts use a logical I/O proxy, called the Protocol Endpoint (PE), to communicate with vVols and virtual disk files that vVols encapsulate. ESXi uses Protocol Endpoints (PE) to establish a data path on demand from virtual machines to their respective vVols.

· Separate the access points from the storage itself.

· Can have fewer access points.

· Existing multi-path policies and NFS topology requirements can be applied to the PE.

· Access points that enables communication between ESXi hosts and storage array systems.

· Compatible with all SAN and NAS Protocols: iSCSI, NFS v3, FC, FCoE.

· A Protocol Endpoint can support any one of these protocols at a given time.

vVols Objects

A virtual datastore represents a storage container in vCenter Server and the vSphere Web Client. vVols are encapsulations of virtual machine files, virtual disks, and their derivatives.

· Virtual machine objects stored natively on the array storage containers

· There are five different types of recognized vVols:

1. Config-vVol - Metadata

2. Data-vVol - VMDKs

3. Mem-vVol - Snapshots

4. Swap-vVol - Swap files

5. Other-vVol - Vendor solution specific

VSC (Virtual Storage Console)

The virtual appliance for Virtual Storage Console (VSC), VASA Provider, and Storage Replication Adapter (SRA) provides end-to-end life cycle management for virtual machines in VMware environments that use NetApp storage systems. It simplifies storage and data management for VMware environments by enabling administrators to directly manage storage within the vCenter Server.

The 9.6 release of the virtual appliance for Virtual Storage Console (VSC), VASA Provider, and Storage Replication Adapter (SRA) provides the combined features of VSC, VASA Provider, and SRA in a single deployment.

Support for ONTAP 9.6:

The 9.6 release of the virtual appliance for Virtual Storage Console (VSC), VASA Provider, and Storage Replication Adapter (SRA) supports ONTAP 9.6.

Availability of VVol reports:

Starting with the 9.6 release of the virtual appliance for VSC, VASA Provider, and SRA, you can view and export VVol datastore reports. The VVol reports enable you to review and identify potential issues with the VVol datastores and virtual machines in your vCenter Server. The following VVol reports are available:

• VVol Datastore Report

• VVol Virtual Machine Report

Open Virtual Storage Console.

You can see the VSC dash board, from there you can add the storage system and Provision the Datastore.

List the available connected storage systems.

Here you can see the 2 ontap 9.6 clusters.

Now go for the provisioning data store.

Create a New datastore using the following information.

Provisioning Destination Storage System
Type of Datastore
Name of the datastore
Protocol

Now select the Storage Capability Profiles.

Create a new Flex volumes with your required size.

Here I have created 2 Flexvols with 10GB size.

New iscsi vVol datastore created successfully.

Same as you can create a vVol using NFS protol also.

You can expand the datastore also.

You can create a Policies and Profiles also.

Create a new VM storage Policy.

Select the vCenter server and Policy Name.

Now select the Policy Rules.

Now select the compatible storage vVol objects.

You can edit and set this new VM storage policy as a default one for any vVol datastore.

Now deploy a new VM from an existing VM templates.

Now select the vVol data store

VM is created and powered on successfully.

Now login to the new VM.

New vVol reports section.

Performing VM level snapshots and Clones are off-loaded to NetApp Storage.

Take a snapshot of the VM.

Check the storage efficiency from ONTAP system Manager.
All De-Duplication and Efficiency for the vVol datastore.

Take one moew snapshot and see the difference in duplication.

Wednesday, July 31, 2019

NetApp MAX DATA Configuration and Management

NetApp MAX Data is an application acceleration product that uses the latest server-side storage technologies to improve application response time and provide enterprise-class data protection to persistent memory. MAX Data provides a file system for your application data with the performance of Intel Optane DCPMM and the capacity of a NetApp AFF storage array.

MAX FS, the file system for MAX Data, spans the memory and storage tiers in a single file system and intelligently moves (tiers) data between the memory tier and the storage tier based on use. It keeps the hottest, or most frequently accessed, data in the memory tier, and tiers the cooler, or less frequently used, data to the storage tier.

This behavior allows you to accelerate your application working set at memory speed while allowing a much larger total data-set size. MAX Data also provides enterprise data services to Optane DCPMM with snapshots and replication technologies. NetApp, as an industry leader, is the first to offer an application-independent solution that merges the highest local speeds with global data protection

MAX Data is a file system driver, currently available for Linux operating systems that will utilize DRAM and persistent memory in the server as a primary data tier. Persistent memory used is Intel Optane (3D Xpoint) as NVDIMMs or PCIe attached. MAX Data expands the internal persistent memory with a backing storage for the primary data tier that is called the secondary data tier. The secondary data tier is a storage network connected NetApp All Flash Array.

Data stored on the server in the primary data tier is also protected by mirroring to a remote MAX Recovery Server. The MAX Recovery Server is another server with the MAX Data file system driver and persistent memory connect over a network.

The NetApp All Flash Array may be an All Flash FAS (AFF) or an E-Series system. NetApp storage is connected as block storage over a storage network. A LUN from the NetApp storage appears as persistent memory to the application with the MAX Data software managing the backing store transfer between the storage and the server.

Extract the NetApp Max_data package in RedHat Linux 7.x

Go to the package directory and install max_data software using the max_install.sh script.

After successful installation, you can access and configure max_data via browser using https://hostname of linux server:9090

Access via web browser. Now login with linux credentials.

Now the Max Data Configurator page opens and you can manage your max_data.

Using this, you can integrate to ONTAP9, Configure MAX DR servers, MAX data Hosts and Application Storage Provisioning.

First Configure the Host Cluster.

Specify the ONTAP9 Cluster Vserver information with credentials, Protocol and Management SVM IP.

After providing all the information, Now Connect to ONTAP 9 Cluster.

Now connected successfully to ONTAP9 Cluster.

For MAX DR, Add License and Specify the Max recovery server IP address.

Now add the MAX Data Hosts. Provide max_data software installed linux server information.

Now successfully added and listing the MAX data Host.

Next Step to provisioning the Storage from NetApp AFF system.

For that add a Max Data License and Provide the Application Type.

Then provide the number of Lun's with the mount name and size.

Then Set the SnapShot copies for Data Protection as per schedule.

Now Application with the details added successfully.

Now you can go for deploy this.

While it is doing deployment process, it will connect to ONTAP9 and create igroup, LUN's and Map it.

Server will reboot, so you reconnect again.

You can check the configuration Logs also.

Now MAX cluster done successfully and application is in active state.

You manage and monitor capacity, performance and utilization.

Lun's are deployed as a datafiles here.

For data protection, you can create a snapshot for max_data files.

Snapshot created.

If you check in the cluster you can see the snapshot.

Just to check this, delete some files and restore the snapshot to recover the files.

Now restore the data from the snapshot.

Data restored successfully.

Thursday, July 4, 2019

NetApp ONTAP 9.6 New Features

NetApp announced ONTAP 9.6 with new features and functions:

All version of ONTAP will be “long term supported”
New version of ONTAP System Manager that is based on REST APIs & New version on Active IQ Unified Manager (Re-Branding)
Simpler FlexGroup management
Adaptive QoS support for NVMe/FC (maximums)
Additional host support for NVMe
FabricPool tiers to Google and Alibaba clouds
FlexCache extended to use with Cloud Volumes ONTAP
FlexGroups on MetroCluster.
Larger ONTAP Select “Premium XL” license
Over-the-wire encryption with SnapMirror and FlexCache
Per tenant / SVM encryption key management
Aggregate-level encryption which enables aggregate level dedupe with NVE
Entry level AFF and FAS MetroCluster over IP
Easy deploy plugin for ONTAP Select in VMware
SnapMirror Synchronous support for NFSv4, SMB2 & 3
Self Encrypting Drives (SED) NVMe SSDs