NSX-T - Disk Space Requirements Not Met
Upgrading NSX-T in the lab, the Host Pre-Check came back with:
Disk space requirements not met. Please ensure 180MB free space is available in tmp partition to proceed with upgrade.
The usual reaction is to check disk usage on ESXi:
1[root@esxi:~] df -h
2Filesystem Size Used Available Use% Mounted on
3NFS 7.0T 5.2T 1.8T 74% /vmfs/volumes/NFS2019
4NFS 7.0T 5.2T 1.8T 74% /vmfs/volumes/NFS1
5vfat 4.0G 208.9M 3.8G 5% /vmfs/volumes/BOOTBANK1
6vfat 4.0G 202.9M 3.8G 5% /vmfs/volumes/BOOTBANK2
7vsan 1.8T 1.3T 532.9G 71% /vmfs/volumes/vsanDatastore
8[root@esxi:~]
But where is /tmp mounted?
I'm booting from a USB stick, which sets /tmp to ramdisk.
To view the ramdisk, use vdf -h
1[root@esxi:~] vdf -h
2[...Output omitted...]
3-----
4Ramdisk Size Used Available Use% Mounted on
5root 32M 6M 25M 21% --
6etc 28M 760K 27M 2% --
7opt 32M 1M 30M 5% --
8var 48M 13M 34M 28% --
9tmp 256M 88M 167M 34% --
10iofilters 32M 0B 32M 0% --
11shm 1024M 0B 1024M 0% --
12crx 1024M 0B 1024M 0% --
13configstore 32M 132K 31M 0% --
14configstorebkp 32M 132K 31M 0% --
15hostdstats 553M 5M 547M 0% --
16nestdb 512M 34M 477M 6% --
17nsx-idps 64M 2M 61M 3% --
18[root@esxi:~]
And here we are. /tmp has 167M available.
For more verbose output:
1[root@esxi:/var/log] esxcli system visorfs ramdisk list
2Ramdisk Name System Include in Coredumps Reserved Maximum Used Peak Used Free Reserved Free Maximum Inodes Allocated Inodes Used Inodes Mount Point
3-------------- ------ -------------------- --------- ----------- --------- ---------- ----- ------------- -------------- ---------------- ----------- -----------
4root true true 32768 KiB 32768 KiB 6916 KiB 6920 KiB 78 % 78 % 18944 6848 6837 /
5etc true true 28672 KiB 28672 KiB 760 KiB 828 KiB 97 % 97 % 4096 1024 848 /etc
6opt true true 0 KiB 32768 KiB 1656 KiB 1656 KiB 94 % 0 % 8192 1024 751 /opt
7var true true 5120 KiB 49152 KiB 13888 KiB 13952 KiB 71 % 0 % 8192 1312 1286 /var
8tmp false false 2048 KiB 262144 KiB 90948 KiB 111736 KiB 65 % 0 % 8192 288 99 /tmp
9iofilters false false 0 KiB 32768 KiB 0 KiB 0 KiB 100 % 0 % 10240 32 1 /var/run/iofilters
10shm false false 0 KiB 1048576 KiB 0 KiB 0 KiB 100 % 0 % 512 32 1 /var/run/shm
11crx false false 0 KiB 1048576 KiB 0 KiB 0 KiB 100 % 0 % 8192 32 2 /var/run/crx
12configstore false false 32768 KiB 32768 KiB 132 KiB 160 KiB 99 % 99 % 8192 32 3 /etc/vmware/configstore
13configstorebkp false false 32768 KiB 32768 KiB 132 KiB 136 KiB 99 % 99 % 8192 32 3 /var/lib/vmware/configstore/backup
14hostdstats false false 0 KiB 566272 KiB 5200 KiB 5200 KiB 99 % 0 % 8192 32 4 /var/lib/vmware/hostd/stats
15nestdb false false 0 KiB 524288 KiB 35512 KiB 44556 KiB 93 % 0 % 8192 32 10 /var/lib/vmware/nsx/nestdb/db
16nsx-idps false false 0 KiB 65536 KiB 2324 KiB 2324 KiB 96 % 0 % 8192 32 2 /etc/nsx-idps/rules
17[root@esxi:/var/log]
Simply by deleting files from /tmp you could fix the problem. But which files?
For ESXi 7.x installed on a USB stick, you'll have the scratch partition at: /tmp/_osdataxxxxxxx/. But don't delete files that you don't know what they do. To find out what folder is using up disk space, run [root@esxi:~] du -h
In my case, the NSX nestdb had grown quite large. Rather than attempt to delete enough benign files, a reboot resolved the issue.
As a comparison in disk space usage:
| Before Reboot | After Reboot |
|---|---|
| [root@esxi:/var] du -h 34.7M ./lib/vmware/nsx/nestdb/db 34.7M ./lib/vmware/nsx/nestdb 34.7M ./lib/vmware/nsx 39.9M ./lib/vmware 51.9M ./lib 13.5M ./log 68.3M . [root@esxi:/var] |
[root@esxi:/var] du -h 20.0K ./lib/vmware/nsx/nestdb/db 24.0K ./lib/vmware/nsx/nestdb 28.0K ./lib/vmware/nsx 2.8M ./lib/vmware 14.8M ./lib 708.0K ./log 18.3M . [root@esxi:/var] |
And to confirm /tmp:
1[root@esxi:~] vdf -h
2[...Output omitted...]
3Ramdisk Size Used Available Use% Mounted on
4root 32M 6M 25M 20% --
5etc 28M 728K 27M 2% --
6opt 32M 1M 30M 5% --
7var 48M 696K 47M 1% --
8tmp 256M 3M 252M 1% --
9iofilters 32M 0B 32M 0% --
10shm 1024M 0B 1024M 0% --
11crx 1024M 0B 1024M 0% --
12configstore 32M 136K 31M 0% --
13configstorebkp 32M 136K 31M 0% --
14hostdstats 553M 2M 550M 0% --
15nestdb 512M 16K 511M 0% --
16nsx-idps 64M 0B 64M 0% --
17[root@esxi:~]
Now there's enough space to continue the NSX-T upgrade.