Hello All,
After installing some updates (12th December) we noticed that one of our AD servers was running out of storage space and RAM utilisation had reached 90% - on investigation we found that the NTDS database was 49GB in size!
Environment:
- 2x Windows Server 2008 R2 running as Virtual Machines; only two domain/DNS servers in the architecture
- Each has 12GB RAM and 8 cores
- 200 AD users, nothing special...
- Tombstone lifetime 180 Day default
So Far:
I have cloned the VMWare servers into an isolated network to play with - I have now spent 6 days on it to no avail (well; I got it down to 16GB with some serious hacks...). Here are some of the key things I have tried in many combinations:
- When the server reboots the RAM utilisation slowly creeps back up to 90% utilisation for lsass.exe
- I have tried offline defrag; this actually makes the database bigger by 2GB...
- Tried 'Semantic Database Analysis' which outputs the following (from this it looks like there are a lot of objects which are deleted but not removed):
Summary:
Active Objects: 5930
Phantoms: 4580
Security descriptor summary:
SD count: 181
Total SD size before single-instancing: 20715440 Kb
Total SD size after single-instancing: 182 Kb
- Tried lowering the 'tombstone lifetime' to 2 days; obviously didn’t change the size after online defrag - after offline defrag the file size reduced to 16GB (on one occasion) and 30GB (on another occasion). So there are definitely a lot of dead objects...
- Running 'esentutl /ms c:\windows\ntds\ntds.dit' reveals:
------------------------------------------------------------------------------------------------
C:\Users\Administrator.server>esentutl /ms c:\Windows\NTDS\ntds.dit
Extensible Storage Engine Utilities for Microsoft(R) Windows(R)
Version 6.1
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Database: c:\Windows\NTDS\ntds.dit
******************************** MSysDefrag DUMP ***********************************
ObjidFDP: 2
OLDStatus: NULL
PassStartDateTime: 05/01/2013 01:19 (0x1cdeae2b91886e3)
PassElapsedSeconds: 3324
PassInvocations: 1
PassPagesVisited: 3986066
PassPagesFreed: 372
PassPartialMerges: 90
TotalPasses: 1204
TotalElapsedSeconds: 7387
TotalInvocations: 1210
TotalDefragDays: 1204
TotalPagesVisited: 199486704
TotalPagesFreed: 2774
TotalPartialMerges: 38345
******************************** SPACE DUMP *****************************************
Name Type Owned(MB) %OfDb %OfTable Avail(MB) Avail
%Tbl
=====================================================================================
c:\Windows\NTDS\ntds.di Db 47998.876 100.00% 16405.461
datatable Pri 31589.336 65.81% 100.00% 348.227 1.10%
Ancestors_index Idx 790.329 1.65% 2.50% 0.180 0.00%
DRA_USN_CREATED_ind Idx 386.055 0.80% 1.22% 0.711 0.00%
INDEX_00000000 Idx 475.383 0.99% 1.50% 1.376 0.00%
INDEX_00020013 Idx 259.149 0.54% 0.82% 11.899 0.04%
INDEX_00020078 Idx 256.446 0.53% 0.81% 4.110 0.01%
INDEX_00090001 Idx 3057.274 6.37% 9.68% 1.422 0.00%
INDEX_00090002 Idx 1092.477 2.28% 3.46% 85.141 0.27%
nc_guid_Index Idx 1097.086 2.29% 3.47% 4.469 0.01%
PDNT_index Idx 3059.055 6.37% 9.68% 1.329
0.00%
Note: Some small tables/indices were not printed (use /v option to see those smaller than 0.5% of the database).
-------------------------------------------------------------------------------------
Enumerated 12 Tables ( 108 Internal Trees, 2 Long Value Trees, 156 Secondary Indices )
Pages 6143856 ( 3981829 Used (64.8%), 2162027 Available (35.2%) )
Note: This database is over 20% empty, an offline defragmentation can be used to shrink the file.
Operation completed successfully in 1.498 seconds.
------------------------------------------------------------------------------------------------
Although a production system; I am happy to try anything on a cloned VM - if successful I am willing to schedule downtime to fix.
Any help would be greatly appreciated
Thanks
Matthew