Monday, October 7, 2019

Deploying NetApp HCI using NetApp Deployment Engine (NDE)




NetApp HCI:



NetApp HCI provides both storage and compute resources, combining them to build a VMware vSphere environment backed by the capabilities of NetApp Element software. You can integrate NetApp HCI seamlessly with other products in the NetApp Data Fabric, creating a complete solution for your datacenter.
After successful deployment of NetApp HCI, you manage the system from the VMware vSphere Web Client. The NetApp Element Plug-in for vCenter Server (also referred to as the vCenter Plug-in, or VCP) is used to manage storage resources, including datastores, volumes, Quality of Service, storage cluster components, and data protection. Compute nodes appear as ESXi hosts and you can manage them in vSphere.






Minimal Configuration:





Deploying NetApp HCI
The NetApp Deployment Engine enables you to quickly deploy NetApp HCI. During deployment, you can let the NetApp Deployment Engine automatically set many of the networking configuration details for you. After deployment, NetApp HCI will be ready to serve highly available compute and storage resources in a production environment.

The NetApp Deployment Engine (NDE) greatly simplifies the configuration of NetApp HCI. By presenting a few consolidated web forms to fill out, the NDE quickly gathers the needed information and automates the deployment of the following HCI components:
• Element storage cluster.
• vSphere ESX hypervisors.
• vSphere datastores hosted on the Element cluster.
• vSphere vCenter server instance (or, optionally, it utilizes an existing vCenter instance).

• NetApp Management Node (for managing the Element cluster, including the vCenter plugin).


Deployment Process:

Accessing the NetApp Deployment Engine


To deploy NetApp HCI, you need to access the NetApp Deployment Engine on one of the NetApp H-Series storage nodes via the IPv4 address assigned to the Bond1G interface, which is the logical interface that combines ports A and B for storage nodes. This storage node becomes the controlling storage node for the deployment process. Depending on your environment, you need to either configure the IPv4 address or retrieve it from one of the storage nodes.

Access the NDE 






Check and Accept the prerequisites.
Power on all network switch and compute and storage nodes.






Accept the End User License Agreement for the following:

1. NetApp EULA
2. VMware EULA




Next provide the vSphere Config details

version, vCenter Server  FQDN and User Credentials.





Provide the Credentials for Storage Node, vCenter and Management Node.




Next provide the Network Topology config details for compute and storage nodes.

If you are going to use distributed switch then select that option.








This will do the inventory of your compute and storage nodes, then select the nodes for your configuration.







Then Network settings, provide the DNS Server and NTP server details.

Then launch the easy form to enter fewer network settings.




Set the vCenter and Management Node Networking details.



Set the Compute Node network details.



Storage Node Network details.





Easy form Network Settings.



Once you provide all the network details then apply this.




 Final Review and Summary.





Then start the deployment Process.





The NDE will now automate the following tasks:

• Create a four-node Element cluster.
• Create two Element volumes for use as vSphere datastores.
• Create two ESX hypervisor hosts.
• Add the ESX iscsi initiators to the Element configuration.
• Create an Element Volume Access Group and configure it to include both datastore volumes and both ESX initiators.
• Format the attached volumes (iscsi LUNs) with VMFS.
• Configure vSphere portgroups and uplinks for managing VM traffic, including vCenter and the Management Node.
• Deploy the vSphere vCenter Server Appliance (VCSA) onto an ESX host and use an Element-supported datastore for the VCSA storage.
• Register the two ESX hosts with the vCenter instance.

• Deploy the NetApp Management Node and register it with vCenter.



Now you can monitor using the URL (NDE Monitoring)





Setup Started.




First validate the Deployment data and then starts configuring Network.






Once the Network Config is done then it is setting up NetApp Element clusters.





Then it is setting up ESXi Server.




Now it is setting up vCenter Server.




Finally it is setting up the Management Node to Manage the Element cluster.







Now the installation and setup completed successfully. 

Then we can access and manage using vSphere web client.






Login to vSphere web Client 6.7.




This deployment process automatically creates VMware HA DRS and add the 2 esxi Hosts and deploy NetApp Management Node and vCenter Server.




Now check with NetApp Element configuration and Management Plugins.




Cluster details.



Manage Node details and settings.




Report and iscsi sessions.






Data Store details.



You can do all the data protection from here it self like snapshot and replication.





Login to NetApp Element Management Node and check.












Tuesday, October 1, 2019

Configuring VMware vSphere vVol DataStore - NetApp ONTAP 9.6 & VSC 9.6



What is Virtual Volumes (vVols)?

Eliminate physical containers and enable more flexible consumption of resources with Virtual Volumes (vVols). This integration and management framework for external storage provides finer control at the VM-level, streamlines storage operations, and offers flexibility of choice.
  
vVols is an integration and management framework that virtualizes SAN/NAS arrays, enabling a more efficient operational model that is optimized for virtualized  environments and centered on the application instead of the infrastructure. vVols simplifies operations through policy-driven automation that enables more agile storage consumption for virtual machines and dynamic adjustments in real time, when they are needed. It simplifies the delivery of storage service levels to individual applications by providing finer control of hardware resources and native array-based data services that can be instantiated with virtual machine granularity.

vVols are VMDK granular storage entities exported by storage arrays. vVols are exported to the ESXi host through a small set of protocol end-points (PE). Protocol Endpoints are part of the physical storage fabric, and they establish a data path from virtual machines to their respective vVols on demand. Storage systems enable data services on vVols. The results of these data services are newer vVols. Data services, configuration and management of virtual volume systems is exclusively done out-of-band with respect to the data path. vVols can be grouped into logical entities called storage containers (SC) for management purposes. The existence of storage containers is limited to the out-of-band management channel.

vVols and Storage Containers (SC) form the virtual storage fabric. Protocol Endpoints (PE) are part of the physical storage fabric.

By using a special set of APIs called vSphere APIs for Storage Awareness (VASA), the storage system becomes aware of the vVols and their associations with the relevant virtual machines.


Characteristics of vVols:
·         No File System.
·         ESX manages the array through VASA (vSphere APIs for Storage Awareness) APIs.
·         Arrays are logically partitioned into containers, called Storage Containers.
·         Virtual machine disks, called vVols, stored natively on the Storage Containers.
·         IO from ESXi host to the storage array is addressed through an access point called, Protocol Endpoint (PE).
·         Data Services are offloaded to the array. Snapshot, Replication, Encryption.
·         Managed through storage policy-based management (SPBM) framework.

VASA (vSphere APIs for Storage Awareness) APIs (VP)

A vVols storage provider, also called a VASA provider is a software component that acts as a storage awareness service for vSphere. The provider mediates out-of-band communication between the vCenter Server and ESXi hosts on one side and a storage system on the other.
·         Software component developed by Storage Array Vendors.
·         ESX and vCenter Server connect to VASA Provider.
·         Provides Storage awareness services.
·         Single VASA Provider can manages multiple arrays.
·         Supports VASA APIs exported by the ESXi host.
·         VASA Provider can be implemented within the array’s management server or firmware.
·         Responsible for creating vVols.
Storage Containers (SC)
Unlike traditional LUN and NFS based vSphere storage, the vVols functionality does not require pre-configured volumes on a storage side. Instead, vVols uses a storage container, which is a pool of raw storage capacity or an aggregation of storage capabilities that a storage system can provide to vVols.
·         Logical storage constructs for grouping of vVols.
·         Logically partition or isolate virtual machines with diverse storage needs and requirement.
·         A single Storage Container can be simultaneously accessed via multiple Protocol Endpoints.
·         Desired capabilities are applied to the Storage Containers.
·         VASA Provider discovers Storage Container and reports to the vCenter Server.
·         Any new virtual machines that are created are subsequently provisioned in the Storage Container.
Protocol Endpoint (PE)

Although storage systems manage all aspects of vVols, ESXi hosts have no direct access to vVols on the storage side. Instead, ESXi hosts use a logical I/O proxy, called the Protocol Endpoint (PE), to communicate with vVols and virtual disk files that vVols encapsulate. ESXi uses Protocol Endpoints (PE) to establish a data path on demand from virtual machines to their respective vVols.
·         Separate the access points from the storage itself.
·         Can have fewer access points.
·         Existing multi-path policies and NFS topology requirements can be applied to the PE.
·         Access points that enables communication between ESXi hosts and storage array systems.
·         Compatible with all SAN and NAS Protocols: iSCSI, NFS v3, FC, FCoE.
·         A Protocol Endpoint can support any one of these protocols at a given time.
vVols Objects

A virtual datastore represents a storage container in vCenter Server and the vSphere Web Client. vVols are encapsulations of virtual machine files, virtual disks, and their derivatives.
·         Virtual machine objects stored natively on the array storage containers
·         There are five different types of recognized vVols:
1.     Config-vVol - Metadata
2.     Data-vVol - VMDKs
3.     Mem-vVol - Snapshots
4.     Swap-vVol - Swap files
5.     Other-vVol - Vendor solution specific

VSC (Virtual Storage Console)

The virtual appliance for Virtual Storage Console (VSC), VASA Provider, and Storage Replication Adapter (SRA) provides end-to-end life cycle management for virtual machines in VMware environments that use NetApp storage systems. It simplifies storage and data management for VMware environments by enabling administrators to directly manage storage within the vCenter Server.

The 9.6 release of the virtual appliance for Virtual Storage Console (VSC), VASA Provider, and Storage Replication Adapter (SRA) provides the combined features of VSC, VASA Provider, and SRA in a single deployment.

Support for ONTAP 9.6:

The 9.6 release of the virtual appliance for Virtual Storage Console (VSC), VASA Provider, and Storage Replication Adapter (SRA) supports ONTAP 9.6.

Availability of VVol reports:

Starting with the 9.6 release of the virtual appliance for VSC, VASA Provider, and SRA, you can view and export VVol datastore reports. The VVol reports enable you to review and identify potential issues with the VVol datastores and virtual machines in your vCenter Server. The following VVol reports are available:
• VVol Datastore Report
• VVol Virtual Machine Report

Login to vSphere web client 6.7.

Open Virtual Storage Console.



You can see the VSC dash board, from there you can add the storage system and Provision the Datastore.




List the available connected storage systems.

Here you can see the 2 ontap 9.6 clusters.




Now go for the provisioning data store.




Create a New datastore using the following information.

Provisioning Destination Storage System
Type of Datastore
Name of the datastore
Protocol




Now select the Storage Capability Profiles.




Create a new Flex volumes with your required size.



Here I have created 2 Flexvols with 10GB size.





New iscsi vVol datastore created successfully.




Same as you can create a vVol using NFS protol also.








You can expand the datastore also.






Starting with the 9.6 release of the virtual appliance for VSC, VASA Provider, and SRA, you can view and export VVol datastore reports. The VVol reports enable you to review and identify potential issues with the VVol datastores and virtual machines in your vCenter Server.

The following VVol reports are available:

• VVol Datastore Report
• VVol Virtual Machine Report


You can create a Policies and Profiles also.




Create a new VM storage Policy.




Select the vCenter server and Policy Name.



Now select the Policy Rules.



Now select the compatible storage vVol objects.






You can edit and set this new VM storage policy as a default one for any vVol datastore.






 Now deploy a new VM from an existing VM templates.








Now select the vVol data store




VM is created and powered on successfully.




Now login to the new VM.




New vVol reports section.


Performing VM level snapshots and Clones are off-loaded to NetApp Storage.

Take a snapshot of the VM.







Check the storage efficiency from ONTAP system Manager.
All De-Duplication and Efficiency for the vVol datastore.




Take one moew snapshot and see the difference in duplication.