LiveZilla Live Chat Software

Share NTFS USB Hard Drive on VMware vSphere/ESX 4

Written by Damien Ramé on . Posted in Best Practices

The other day, I was discussing virtualization in email security (for example, anti spam virtual appliance). One of the drawbacks of ESX (and ESXi) is the lack of support for external data storage (non Data Store). Of course, you can add a multitude of data stores, local and remote, and it is THE way to go.

But I needed to be able to backup my virtual appliances to an external hard drive using NTFS (so that it can be read on my Windows machines). My VMware physical server is based off a simple (and cheap) quad core AMD and not a Xeon, so I do not have the USB Passthrough technology to virtualize the USB hub. But I needed to have a way to access my external NTFS USB hard drive from either the host or any of the guests. And I needed to this locally because we’re talking hundreds of Gigabytes of data I was not willing to go through my Gigabit network. USB-IP adapters (also called Network USB hubs) are not an option because they are still using the network and generally offer poor performance/dollar.

It is possible. I’m not saying it is the best solution, or the most stable solution. But I have been running this on my ESX host (with 6 VMs) for 2 months now, without a single glitch, so I thought I’d share this howto (source material in reference below).

STEP 1: INSTALL THE NTFS DRIVER ON ESX 4

First, we will download the RPM for the NTFS-3G driver, and install it:

[root@localhost ~]# mkdir ntfstemp
[root@localhost ~]# cd ntfstemp
[root@localhost ~]# lwp-download http://marush.com/wp-content/uploads/2009/04/ntfs-3g-200944-el5i686.rpm
[root@localhost ~]# rpm -i ntfs-3g-200944-el5i686.rpm

Following that, we will mount the USB hard drive. Assuming it is already plugged in the USB port and powered on, it should be mapped to a device name.

[root@localhost ~]# fdisk -l
This will give you a list of drives and partitions. In this example, I will assume the drive is mapped to /dev/sdj with a single partition (/dev/sdj1) as illustrated in the partial fdisk output below:
Disk /dev/sdj: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdj1 1 60801 488384001 7 HPFS/NTFS

To mount the USB hard drive, you need to create a mountpoint (where the drive contents will be mapped to). Let’s say we want to mount it to /mnt/seagate500gb

[root@localhost ~]# ntfs-3g /dev/sdj1 /mnt/seagate500gb

The driver will return the following warning. Read it carefully.

WARNING: Deficient Linux kernel detected. Some driver features are
not available (swap file on NTFS, boot from NTFS by LILO), and
unmount is not safe unless it’s made sure the ntfs-3g process
naturally terminates after calling ‘umount’. If you wish this
message to disappear then you should upgrade to at least kernel
version 2.6.20, or request help from your distribution to fix
the kernel problem. The below web page has more information:
http://ntfs-3g.org/support.html#fuse26

Next, we will verify the drive is mounted properly:

[root@localhost ~]# mount | grep sdj1
/dev/sdj1 on /mnt/seagate500gb type fuse (rw,allow_other)

It’s all good, let’s see the contents of the drive with a ‘ls’ command:

[root@localhost ~]# ls -l /mnt/seagate500gb

drwxrwxrwx 1 root root 0 Apr 18 10:47 $RECYCLE.BIN
drwxrwxrwx 1 root root 0 Mar 27 15:33 RECYCLER
drwxrwxrwx 1 root root 0 Mar 27 22:16 System Volume Information

You can then copy some files to the USB drive for testing. I was able to get very good speeds out of it (50 MB/s if I recall correctly).

STEP 2: INSTALL THE SAMBA SERVER ON ESX 4

Now that the ESX host can R/W access the external NTFS drive, we need to make that available for the guests (and the rest of the network in this case, but you can restrict access by IP and by using a username/password).

You first need to download a few RPM from the CentOS 5.2 distribution (which is what ESX is based on), specifically:

  • samba-common-3.0.28-1.el5_2.1.x86_64.rpm
  • cups-libs-1.2.4-11.18.el5_2.3.x86_64.rpm
  • samba-3.0.28-1.el5_2.1.x86_64.rpm

Locating these can be a problem as 5.2 has been replaced by newer revisions and many mirrors tend to keep only the latest revision. I suggest Googling the RPM file (it needs to be x86_64 RPM with the proper revision, I haven’t tested any newer version). Or you can go to CentOS mirror list and search for version 5 in 64-bit here:
http://www.centos.org/modules/tinycontent/index.php?id=30

Once they are downloaded, you need to install/upgrade these. Ignore any warning or message saying that the package is already installed in the commands below:

[root@localhost ~]# rpm -Uvh samba-common-3.0.28-1.el5_2.1.x86_64.rpm
[root@localhost ~]# rpm -Uvh—nodeps cups-libs-1.2.4-11.18.el5_2.3.x86_64.rpm
[root@localhost ~]# rpm -Uvh samba-3.0.28-1.el5_2.1.x86_64.rpm

Next, you need to tell ESX to open the incoming port 445 on the host so that other computers (virtual or physical) can connect to the Samba server. Open the firewall in ESX using command-line tool:

esxcfg-firewall -o 445,tcp,in,smb-server

You can also use the vSphere client instead.

STEP 3: SHARE THE EXTERNAL USB HARD DRIVE

Finally, you need to tell Samba to share your mounted USB drive (mounted on /mnt/seagate500gb in this example). Do a search on “vi” if you are not used to it. It is an old text editor in Unix and is everything but user friendly. Alternately, you can edit the file on a remote computer with a GUI and just copy the file back to its location (/etc/samba/smb.conf)

[root@localhost ~]# vi /etc/samba/smb.conf

In the [global] section, you can (optionally) change these:

workgroup = WORKGROUP_NAME
server string = ESX 4 Server

At the end of the file, add this section:

# Seagate 500GB
[Seagate 500gb]
comment = Seagate 500GB Hard Drive
browseable = yes
writable = yes
valid users = root,otherusername
path = /mnt/seagate500gb

I also added these so that I can have access to the data store files directly:

# VMFS
[vmfs]
comment = vmfs
browseable = yes
writable = yes
valid users = root,otherusername
path = /vmfs/volumes

# VM Images
[vmimages]
comment = vmimages
browseable = yes
writable = yes
valid users = root,otherusername
path = /vmimages

Now, you need to create a Samba password for root as it needs to be synchronized with the ESX username DB (for security purposes, it would be even better to use “otherusername” for the Samba password sync, and for the share in smb.conf – the “valid users” option), start the Samba service and restart the Firewall.

[root@localhost ~]# smbpasswd -a root
[root@localhost ~]# /etc/init.d/smb start
[root@localhost ~]# /etc/init.d/firewall restart

Test your new external NTFS USB hard drive share on your Windows clients and/or guest VMs and once you have everything working, you can configure Samba to automatically start when the ESX server is booted (to avoid having to manually start the Samba server, which is still a valid option if you want to have it up and running occasionally only):

[root@localhost ~]# chkconfig—level 3 smb on

Assuming you’re using the automatic start up of Samba, it would be also a good idea to automatically mount your NTFS hard drive at startup (see /etc/fstab for more information).

Have fun. And remember the NTFS-3G warning above.

REFERENCES

 

Tags: , , , , , , ,

Trackback from your site.

Damien Ramé

Damien Ramé

Damien started down the geek road when he got his first computer at age 10: an Atari 400 with a 1.8MHz 6502 CPU, 8K of RAM, no storage (only a true geek includes those stats!). And he hasn’t looked back since: this Software Engineer has administered Linux and Windows servers, developed high- and low-level software and firmware code in a variety of languages, worked in telecommunications, networking and IT, and… ended up getting an MBA while developing a passion for marketing. He’s now a tech-savvy marketer who can talk the talk and walk the walk.

Comments (2)

  • Email Security Matters: A Look Back at the Year | Anti-Spam

    |

    [...] 1-Share NFTS Harddrive on VMware vSphere/ESX 4: One of the drawbacks of ESX (and ESXi) is the lack of support for external data storage (non Data Store). Of course, you can add a multitude of data stores, local and remote, and it is THE way to go. [...]

    Reply

  • aa2k

    |

    Thanks for your great guide!!! … I just finished installing it on ESX 4.1, but I had to adjust a few things to make it work with the new version, some updated packages I needed to install, just google the package name and its easy to find:

    Follow same steps from original poster, if you want to install dependencies for cups-libs, install these first:

    The package “samba-common-3.0.28-1.el5_2.1.x86_64.rpm” is already installed on the new version esx 4.1, to check its there just run “rpm -q samba-common”.

    Install the following 3 packages/dependencies ONLY if you want to install cups-libs with dependencies, otherwise use the –nodeps option when installing cups-libs:

    rpm -Uvh libjpeg-6b-37.x86_64.rpm

    rpm -Uvh libpng-1.2.10-7.1.el5_5.3.x86_64.rpm

    rpm -Uvh libtiff-3.8.2-7.el5_5.5.x86_64.rpm

    If using with the mentioned dependencies install:

    rpm -Uvh cups-libs-1.3.7-11.el5_4.5.x86_64.rpm

    If not using the dependencies install :

    rpm -Uvh –nodeps cups-libs-1.3.7-11.el5_4.5.x86_64.rpm

    rpm -Uvh perl-Convert-ASN1-0.20-1.1.noarch.rpm (this package is needed now)

    rpm -Uvh samba-3.0.33-3.15.el5_4.1.x86_64.rpm

    To set up samba and use it you’ll need to open extra ports for this version (4.1):

    1. Open the following firewall ports in ESX:

    esxcfg-firewall -o 445,tcp,in,smb-server (as original poster, if you do this port only it works but you wont be able to see the samba shares on Windows Networks)

    Open these ports as well and samba shares will be visable on Windows Network and other Unix boxes:

    esxcfg-firewall -o 445,tcp,out,smb-server

    esxcfg-firewall -o 445,udp,in,smb-server

    esxcfg-firewall -o 445,udp,out,smb-server

    esxcfg-firewall -o 137:139,tcp,in,smb-server

    esxcfg-firewall -o 137:139,udp,in,smb-server

    esxcfg-firewall -o 137:139,tcp,out,smb-server

    esxcfg-firewall -o 137:139,udp,out,smb-server

    2. Edit the smb.conf as noted on the original poster:

    3. Follow same steps as original poster.

    NOTE: You may see these warning messages after installing some packages, just ignore them, everyting works.

    /sbin/ldconfig: /usr/lib/libkrb4.so.2 is not a symbolic link

    /sbin/ldconfig: /usr/lib64/libkrb4.so.2 is not a symbolic link

    To query firewall open ports run:

    esxcfg-firewall -q

    In my setup I had an Ubuntu server acting as WINS server, so on this ESX server under “smb.conf” I added the option of “wins server = <my ubuntu server ip>”, not sure if it matters but I mention it.

    Cheers! and thanks for such a great guide!

    Reply

Leave a comment

Whitepaper Downloads

  • Spam Industry terms
  • Antispam checklists
  • Tips and tricks
  • In depth research analysis

Download

Customer Quote

We bought ModusGate to improve our hosting offers and to attract customers.

    Régis Holstein
    TDS