VMware ESX Tutorial

What is Virtualization?

Virtualization is a general and ambiguous term that typically means to run multiple instances of something inside something that was intended to only run a single instance.

How do we get virtualization?

Well there is no. of options available for this, but VMware, Citrix and Microsoft are front runners in domain. Here are the products from above for the same.

1. VMware ESX and VMware server
2. Citrix Xen
3. Microsoft Hyper-V

VMware Server:

VMware Server is a free virtualization product for Microsoft Windows and Linux servers. It enables you to quickly provision new server capacity or server consolidation by partitioning a Physical server into multiple virtual machines. It is meant for those businesses who want to optimize use of their technology assets using virtualization.

Technology behind VMware Server:

• Hypervisor – 2 (Hosted Architecture)

What is Hosted Architecture?

Hosted architecture is where your virtualization software is installed as an application onto the pre-existing host operating system. This means that your virtualization layer relies on your host operating system for device support and physical resource management. VMware Server is a good example of a hosted architecture.

Hardware and Software support:

1. Any standard x86 compatible or x-86-64compatible personal computer
2. A wide variety of Windows, Linux, Solaris, and other guest operating systems,

Including 64-bit operating systems

3. Two-way Virtual SMP
4. Intel Virtualization Technology (Intel VT)
5. AMD-Virtualization (AMT-V)

Where VMware Server does fits for your environment?

VMware server suits the requirements and demands of small and medium scale business with non-mission critical processes. It can provide following options to the business resources.

1. Virtual Machines for software development:
1. A common environment matching in both configuration and tools for your entire team
2. Quick rollout of new environments and tools
3. Archive of entire development environments for major projects

2. Virtual Machines for software testing:
1. Platform Testing with Snapshots
2. Platform Testing with Persistent and Non-persistent Disks

3. Virtual Machines for Post-release and Application-Maintenance.
1. It can create base image for you development environment
2.  Archive of the development team’s virtual machines used during the development phase
3. Rapid platform availability for quick support tasks and bug investigation

4. Run Windows, Linux, and other operating systems and applications without

Software conflicts.

5. Move virtual machines from one physical host to another without having to

Reconfigure them.

Features of VMware Server:

1. Web-Based Interface

Use VMware Infrastructure Web Access (VI Web Access) to perform host and virtual machine configuration for VMware Server 2.0.

1. Create, configure, and delete virtual machines
2. Add and remove virtual machines from the inventory
3. Perform power operations (start, stop, reset, suspend, and resume) on virtual Machines
4. Monitor the operation of virtual machines
5. Generate a Web shortcut to customize the VI Web Access user interface for users, with the option to limit their view to the console or a single virtual machine
6.  Generate a VMware Remote Console desktop shortcut that allows virtual machine users to interact directly with the guest operating system outside of a Web browser
7. Configure host-wide VMware Server settings

2. VMware Remote Console

VMware Remote Console enables you to interact with the guest operating system

running in a virtual machine. You can run VMware Remote Console on the host or a remote client system. After you install it as a Web browser add-on from VIWeb Access, VMware Remote Console can run independently from VIWeb Access.

VMware Remote Console also allows you to connect and disconnect client CD/DVD

and floppy devices.

3. Memory Support

The maximum amount of memory that can be allocated per virtual machine is

8GB. The amount of memory that can be used by all virtual machines combined is limited only by the amount of memory on the host computer.

4. Number of Network Adapters Supported

It can support total of 10 network adapters for a virtual machine.

5. Quiesced Backups of Virtual Machines on Windows

On Windows hosts, you can enable the VMware VSS Writer, which uses snapshots to maintain the data integrity of applications running inside the virtual machine when you take backups.

6. Support for High-Speed USB 2.0 Devices

If the guest operating system has the appropriate USB 2.0 device drivers, you can use peripherals that require high-speed performance, such as speakers, webcams,

next-generation printers and scanners, fast storage devices, MP3 players, DVD-RW

drives, and high-capacity CD-ROM jukeboxes.

Besides above there are few more such as

·         Additional host operating system support

·         Additional guest operating system support

·         improved 64 bit guest support

·         64 bit sound driver

·         Native 64 bit support on linux

·         Improved VIX-API

·         VMCI socket interface

Bottlenecks:

1. It puts more of your client's eggs in one basket. If the host machine breaks or needs to be taken offline, several virtual servers will go down.
2. Applications like databases that require a lot of disk activity. The prevailing wisdom is that databases should still run on dedicated physical servers.
3. Time-sensitive applications like Voice over IP (VoIP) may also be poor candidates for virtualization.
4. Virtual Machine overload: Application that are low I/O intensive and low utilization are best candidates for virtualization but need to put restriction on no. of virtual machines that can be handled with ease on physical server.

VMware Server at a glance:

Architecture	Hosted
OS Requirements	Windows or Linux
Typical Use Cases	Test & Dev, Production
Dedicated Server Required	No
Centralized Management Option	No
Ease of Use	High
Performance	Good

ESX Server

ESX Server is VMware’s flagship enterprise server virtualization platform.  It comes in two versions – ESX Server and ESXi Server where the latter has no service console and is the thinnest version available. ESX Server has many optional features like VMotion and VMHA and some built-in features like the VMFS file system.  Most end users purchase VMware ESX Server with some set of optional features in a package called VMware Infrastructure. ESX Server is managed by the VMware Infrastructure Client. Its centralized management platform is called Virtual Center.

Figure 1: ESX server in Enterprise (Virtual infrastructure)

What is the need of ESX Server?

If you are an idealist then ESX is just for you. It possesses best approaches to adopt the hardware abstraction and most effective usage of your resources through vast available tools and services so that you can maximize your infrastructure efficiency, reducing operational cost by providing cost effective business continuity.

Technology behind ESX:

·         Hypervisor – 1 (Bare Metal)

Bare Metal means no OS is required because it has its own kernel derived from linux withit provides greater resources for the virtual machines, decreased cost of licensing and increased utilization of servers.

Where ESX server does fit for your environment?

As ESX is enterprise wide solution for virtualization to adopt effective hardware abstraction, it best suits for large enterprises with enormous resources.

Hardware and Software support with Features for ESX:

Architecture:

1. Bare-metal architecture:  VMware ESX inserts a robust virtualization layer directly on the server hardware for near-native virtual machine performance, reliability and scalability.

    

     Fig: Hypervisor (Baremetal)

2. Small Footprint: VMware ESXi’s 32MB disk footprint is a fraction of the size of a general purpose operating system, reducing complexity and providing unmatched security and reliability.

3. Server Integration: VMware ESXi is available built into server hardware as an embedded component, simplifying and speeding deployment of virtualization.

4. CPU virtualization:  Increase server utilization without the risk of critical services being starved for CPU resources. VMware ESX uses intelligent process scheduling and load balancing across available processors to manage the execution of virtual machine processing.

5. Storage in ESX Server:  Leverage high performance shared storage to centralize virtual machine file storage for greater manageability, flexibility and availability.
• Virtual disk files:  Add or delete a VMware ESX server from a VMFS volume without pausing or halting the processing of other instances of VMware ESX
• VMFS cluster file system:  Leverage high performance shared storage to centralize virtual machine file storage for greater manageability, flexibility and availability
• Logical volume manager: Manage the interaction between the physical storage arrays and VMFS with flexibility and reliability
• Raw device mapping: Optionally, map SAN LUNs directly to a virtual machine in order to enable application clustering and array-based snapshot technology while profiting from the manageability benefits of VMFS
• Fiber Channel HBA consolidation:  Share expensive storage network components across many virtual machines while maintaining hardware fault tolerance
• Write-through I/O:  Ensure precise recovery of virtual machines in the event of server failure. Write-through I/O enables virtual machines to have the same recovery characteristics as a physical system running the same operating system.
• Boot from SAN:  Run VMware ESX installations on diskless configurations of blade and rack mount servers by booting from SAN. Simplify backups and disaster recovery by eliminating the need to separately backup local attached server disks

VMFS: (Virtual Machine File System) VMware Virtual Machine File System is a high performance cluster file system which provides storage virtualization that is optimized for virtual machines. Each virtual machine is encapsulated in a small set of files; and VMFS is the default storage management interface for these files on physical SCSI disks and partitions.

Fig: VMFS allows sharing of storage for multiple ESX servers

Features of VMFS:

The technical features of VMFS that make it suitable for use in a virtual environment include:

1.   Automated file system with hierarchical directory structure

2.   Optimization for virtual machines in a clustered environment

3.   Lock management and distributed logical volume management

4.   Dynamic datastore expansion by spanning multiple storage extents

5.   Clustered file system with journal logging for fast recovery

6.   Encapsulation of the entire virtual machine state in a single directory

Benefits of VMFS:

1.   Automated Cluster File System Capability

2.   Optimizes VM Access

3.   Encapsulates the Entire VM State in a Single Directory

4.   Simplifies Provisioning and Administration of VMs

5.   Provides Distributed Infrastructure Services for Multi-ESX Servers

6.   Facilitates Dynamic Growth

7.   Intelligent Cluster Volume Management

8.   Enables HA with Lower Management Overhead

9.   Simplifies Disaster Recovery

VMFS V/S other file systems:

Conventional file systems allow only one server to have read-write access to the same file at a given time. In contrast, VMFS is a cluster file system that leverages shared storage to allow multiple instances of ESX Server concurrent read and write access to the same storage resources. VMFS also has distributed journaling to allow fast and resilient recovery across these multi-server resource pools.

On-disk locking in VMFS ensures that a virtual machine is not powered on by multiple installations of ESX Server at the same time. With VMware HA enabled, if a server fails the on-disk lock for each virtual machine is released, allowing the virtual machine to be restarted on other physical servers. Moreover, VMFS provides the VM snapshot capabilities so necessary for disaster recovery, and is the interface which VMware Consolidated Backup (VCB) leverages to provide proxy backup of virtual environments.

VMFS does not have every feature found today in other CFS and CVM systems. However there is no other CFS/CVM that provides the capabilities of VMFS. Its distributed locking methods forge the link between the VM and the underlying storage resources in a manner that no other CFS or CVM can equal. The unique capabilities of VMFS allow VMs to join a cluster seamlessly, with no management overhead.

6. Networking in ESX Server:  Network virtual machines like physical machines. Build complex networks within a single VMware ESX server or across multiple installations of VMware ESX for production deployments or development and testing purposes.

·         Virtual NICs:  Configure each virtual machine with one or more virtual NICs. Each of those network interfaces can have its own IP address and even its own MAC address. As a result, virtual machines are indistinguishable from physical machines from a networking standpoint

·         Virtual switches:  Create a simulated network within a VMware ESX server with virtual switches that connect virtual machines.

·         Expanded port configuration policies:  Simplify port configuration by utilizing a single configuration object across large groups of ports. The configuration object specifies all information needed to enable a port: NIC teaming policy (now per port instead of per virtual switch), VLAN tagging, Layer 2 security, and traffic shaping.

·         VLAN:  Overlay a logical LAN on top of physical LANs to isolate network traffic for security and load segregation purposes. VMware ESX VLANs are compatible with standard VLAN implementations from other vendors. Modify network configurations without having to change actual cabling and switch setups. VLANs keep broadcast traffic limited to the VLAN, reducing the network load of broadcast packets on other switches and network segments.

Performance and Scalability:

VMware ESX delivers unparalleled performance and scalability. With VMware ESX, even the most resource intensive production applications such as databases, ERP and CRM can be virtualized.

1. Enhanced virtual machine performance:  Benefit from better virtual machine performance in VMware ESX. Performance improvements have been achieved through:
• Networking performance optimization: Reduce the CPU overhead associated with processing network i/
• Support for hardware nested page tables: Optimize memory translation time between guest operating systems and physical memory
• Support for large memory pages: Improve memory access efficiency for guest operating systems and the Hypervisor
• Support for Para virtualized Linux guest operating systems (Linux kernel 2.6.21 onwards). Run higher levels of performance through virtualization-aware operating systems

2. Advanced memory management:

·         RAM over-commitment:  Increase memory utilization by configuring virtual machine memory that safely exceeds the physical server memory. For example, the sum of the memory of all virtual machines running on a server with 8GB physical memory can be 16GB

·         Transparent page sharing:  Utilize available memory more efficiently by storing memory pages identical across multiple virtual machines only once. For example, if several virtual machines are running Windows Server 2003, they will have many identical memory pages. Transparent page sharing consolidates those identical pages into a single memory location.

·         Memory ballooning:  Shift memory dynamically from idle virtual machines to active ones. Memory ballooning artificially induces memory pressure within idle virtual machines, forcing them to use their own paging areas and release memory for active virtual machines.

3. Improved power management:  Lower the data center utility bill with improved power management. VMware ESX enters a low power “halt” state when a CPU is not scheduled.

4. 4-Way Virtual SMP(Symmetric multiple processor):  Enable a single virtual machine to use up to four physical processors simultaneously. VMware ESX extends this unique feature from two to four processors. With 4-way Virtual SMP even the most processor intensive software applications like databases and messaging servers can be virtualized.

5. 64GB RAM for virtual machines: Run the most memory-intensive workloads in virtual machines with a memory limit extended to 64GB.

6. Support for powerful physical server systems:  Take advantage of very large server systems with up to 32 logical CPUs and 256GB RAM for large scale server consolidation and DR projects.

7. Support for up to 128 powered-on virtual machines:  Take advantage of very large server systems for enterprise-class server consolidation and containment with support for up to 128 powered on virtual machines on a single server. 

8. Flexible virtual switches:  Scale up to handle more virtual machines. Virtual switches can be created with any number of ports from 8 to 1016, and up to 248 virtual switches are supported per host.

9. Wake-on LAN:  Enable higher consolidation ratios by allowing virtual machines to go on stand-by mode when not used.

Interoperability:

VMware ESX is the only virtualization platform optimized, rigorously tested and certified across the complete IT stack of servers, storage, operating systems, and software applications allowing for enterprise-wide standardization.

1. Hardware:  VMware ESX has been certified with industry-leading rack, tower and blade servers from Dell, Fujitsu Siemens, HP, IBM, NEC, Sun Microsystems and Unisys as well as servers that conform to Intel white-box standard specifications.

VMware ESXi is integrated into server hardware guaranteeing that virtualization works out of the box.

2. Storage:  VMware ESX is certified with a wide range of storage systems from, Dell, EMC, EqualLogic, Fujitsu, Fujitsu Siemens, HP, Hitachi Data Systems, IBM, NEC, Network Appliance, StorageTek, Sun Microsystems and 3PAR and many other vendors.
• Heterogeneous storage arrays:  Utilize a wide variety of heterogeneous storage devices in the same VMFS volume
• NAS and iSCSI SAN support:  By supporting lower-cost, more easily managed shared storage, VMware ESX further reduces total cost of ownership of IT environments. Advanced VMware Infrastructure features like VMotion and VMware HA are fully supported with NAS and iSCSI environments
• 4GB Fibre Channel SAN support:  Centralize management and configuration of all VMware ESX servers in VirtualCenter.
• Local SATA storage support: Use select servers with local SATA storage to further lower total cost of ownership while consolidating workloads

3. Networking: Use high performance networking such as 10 Gig E and Infiniband with  VMware ESX 3.5 and VMware ESXi 3.5 for the most network intensive workloads

4. Operating systems:  Run any software application in VMware virtual machines.

·         64-bit guest operating system suppor

·         Solaris 10 operating system suppor

·         Windows Vista operating system support

·         Ubuntu guest operating system support

5. Software applications with third party systems management products through Web services APIs provided by the VMware Infrastructure SDK.

6. Support for other virtual machine formats:  VMware ESX can run virtual machines created in non-VMware formats. Using the free VMware Virtual Machine Importer users can run Microsoft® Virtual Server and Virtual PC, and Symantec® LiveState Recovery virtual machines in VMware ESX.

Management:

1. Remote Command Line Interface. Manage VMware ESXi through a remote execution environment that can run VMware ESX command scripts.

2. Advanced manageability and usability features:  VMware ESX enables management of entire virtualized IT environment.

3. SMI-S-Compliant Management Interfaces:  Monitor virtual storage using any standard SMI-S-aware storage management tool.

4. Virtual Infrastructure Client:  Manage VMware ESX, virtual machines, and (optionally) VMware vCenter Server with a common user interface.

5. Virtual Infrastructure Web Access:  Manage VMware ESX with simple Web interface (formerly known as the Management User Interface, or MUI).

6. Virtual machine shortcuts: Enable self-help for end users with direct access to virtual machines through a Web browser.

7. Remote devices:  Install software in a virtual machine running on a server from the CD-ROM of a desktop without leaving your desk.

8. Agent-less Hardware Management with CIM:  The Common Information Model (CIM) provides a protocol for monitoring hardware health and status through VirtualCenter or CIM-compatible 3rd party tools.

Fig: ESX-Virtual Center

 

Resource Optimization:

It defines advanced resource allocation policies for virtual machines to improve service levels to software applications. Establish minimum, maximum, and proportional resource shares for CPU, memory, disk and network bandwidth. Modify allocations while virtual machines are running. Enable applications to dynamically acquire more resources to accommodate peak performance.

1. CPU capacity prioritization: CPU capacity is assigned to virtual machines on a “fair share” basis and CPU resource controls also allow an absolute minimum level of CPU capacity to be provided to critical virtual machines

2. Storage I/O traffic prioritization:  Ensure that critical virtual machines receive priority access to storage devices. I/O traffic from virtual machines to disk can be prioritized on a “fair share” basis.

3. Network Traffic Shaper:  Ensure that critical virtual machines receive priority access to network bandwidth. Network traffic from virtual machines can be prioritized on a “fair share” basis. Network Traffic Shaper manages virtual machine network traffic to meet peak bandwidth, average bandwidth and burst size constraints.

4. Resource Pool:  Aggregate collections of hardware resources virtualized by VMware ESX into unified logical resources that can be allocated to virtual machines on-demand. Resource pools increase flexibility and hardware utilization.

Security:

1. Compatibility with SAN security practices: Enforce security policies with LUN zoning and LUN masking.

2. VLAN tagging: Enhance network security by tagging and filtering network traffic on VLANs. Limit the scope of broadcast domains.

3. Layer 2 network security policies:  Enforce security for virtual machines at the Ethernet layer. Disallow promiscuous mode sniffing of network traffic, MAC address changes, and forged source MAC transmits.

ESX VCB(VMware Consolidated backup):

VCB is a group of Windows command line utilities, installed on a Windows system, that has SAN connectivity to the ESX Server VMFS file system. With VCB, you can perform file level or image level backups and restores of the VM guests, back to the VCB server. 

Fig: ESX-VCB

ESX-Vmotion:

VM guests are able to move from one ESX Server to another with no downtime for the users. What is required is a shared SAN storage system between the ESX Servers and a VMotion license.

Fig: ESX-Vmotion

ESX at a glance:

Architecture	Hypervisor
OS Requirements	None
Typical Use Cases	Production, Test & Dev
Dedicated Server Required	Yes
Centralized Management Option	Yes
Ease of Use	High
Performance	Best

Comparison between VMware ESX Server and VMware Server using V-Mark:

Running VMmark using ESX Server on an HP DL585 with four 2.2GHz dual-core processors and then ran VMmark on VMware Server using a similar HP DL585 with four 2.4 GHz dual-core processors. The result is as below:

These results show that ESX Server not only achieves higher throughput than VMware Server for a single VMmark tile (6 workload VMs) but also exhibits better scalability when a second tile is added