Retrieve All Members of Large AD Groups - active-directory

Working with an Microsoft Active Directory and Unboundid SDK and there is a group with >29k members.
I am trying to utilize the range values to get all the groups, but can not determine when the end has been reached.
I am using this method: (Updated to working code)
public static List<String> getAttributeRangeBasedSearch(LDAPConnection ldc, String basedn, String filter, int step, String return_attribute) throws LDAPException
{
List<String> allValues = new ArrayList<String>();
// initialize counter to total the group members and range values
int allvalues = 0;
int start = 0;
// int step = 1000;
int finish = step - 1;
boolean finallyFinished = false;
String range;
// loop through the query until we have all the results
while (!finallyFinished)
{
range = start + "-" + finish;
String currentRange = return_attribute + ";Range=" + range;
String range_returnedAtts[] = { currentRange };
SearchRequest searchRequest = new SearchRequest(basedn, SearchScope.BASE, filter, range_returnedAtts);
List<SearchResultEntry> rangedEntries = ldc.search(searchRequest).getSearchEntries();
for (Iterator<SearchResultEntry> iterator = rangedEntries.iterator(); iterator.hasNext();)
{
SearchResultEntry searchResultEntry = iterator.next();
Collection<Attribute> allAttribute = searchResultEntry.getAttributes();
for (Iterator<Attribute> attributeIterator = allAttribute.iterator(); attributeIterator.hasNext();)
{
Attribute attribute = attributeIterator.next();
log.debug("---> " + allvalues + ": " + attribute.getName());
if (attribute.getName().endsWith("*"))
{
currentRange = attribute.getName();
finallyFinished = true;
}
String[] attributeBatch = searchResultEntry.getAttributeValues(currentRange);
for (int i = 0; i < attributeBatch.length; i++)
{
allValues.add(attributeBatch[i]);
log.debug("-- " + allvalues++ + " " + attribute.getName() + ":" + attributeBatch[i]);
}
}
}// for SearchResultEntry
start = start + step;
finish = finish + step;
}// finallyFinished
return allValues;
}
Any ideas?
Thanks
-jim

I got things working, but the process is very difficult and currently I am using a hard coded value for the step as this could be dynamically changed formt he default of 1,500 to a hard coded limit of 5,000.
I have not been able to determine the value dynamically. Appears, maybe, that if it is not defined at: CN=Query-Policies,CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,forest root then is must be at defaults, which the default, also varies based on which version of Microsoft Active Directory is being used.
There is also described in MSDN about some sort of control that might help, but no information on how it could be used. Anyone ever use this?
LDAP policies are specified using the lDAPAdminLimits attribute.
The lDAPAdminLimits attribute of a queryPolicy object is a multivalued string where each string value encodes a name-value pair. In the encoding, the name and value are separated by an "=". For example, the encoding of the name "MaxActiveQueries" with value "0" is "MaxActiveQueries=0". Each name is the name of an LDAP policy, and the value is a value of that policy.
There can be multiple queryPolicy objects in a AD Forest. A DC determines the queryPolicy object that contains its policies according to the following logic:
If the queryPolicyObject attribute is present on the DC's nTDSDSA object, the DC uses the queryPolicy object referenced by it.
Otherwise, if the queryPolicyObject attribute is present on the nTDSSiteSettings object for the Active Directory Site to which the DC belongs, the DC uses the queryPolicy object referenced by the Active Directory Site.
Otherwise, the DC uses the queryPolicy object whose DN is "CN=Default Query Policy,CN=Query-Policies" relative to the nTDSService object (for example, "CN=Default Query Policy, CN=Query-Policies, CN=Directory Service, CN=Windows NT, CN=Services" relative to the root of the config NC).

This one can retrieve and store in textfile any number of users. Moreover it will not finish in infinite loop if group is empty
$myGroup = [string]$args[0];
$myGroup = $myGroup.replace(" ",",");
$group = [adsi]("LDAP://$($myGroup)");
$from = 0
$all = $false
$members = #()
while (! $all) {
trap{$script:all = $True;continue}
$to = $from + 999
$DS = New-Object DirectoryServices.DirectorySearcher($Group,"(objectClass=*)","member;range=$from-$to",'Base')
$members += $ds.findall() | foreach {$_.properties | foreach {$_.item($_.PropertyNames -like 'member;*')}}
if($from -gt $members.count){
break;
}
$from += 1000
}
$currentExecuting = (Get-Item $MyInvocation.MyCommand.Path)
$group.sAMAccountName
$members | measure-object
$members > "$($currentExecuting.Directory)\$($group.sAMAccountName).txt"
usage:
getADGroupMembers.ps1 CN=groupName,OU=myOrgUnit,DC=contoso,DC=com

Here is a very good code example where you can get all members of a group by ranges. It handles the case when you're on the last range too. You can also transform this method in a paginated request. Have a look. It helped me.
try
{
DirectoryEntry entry = new DirectoryEntry("LDAP://CN=My Distribution List,OU=Distribution Lists,DC=Fabrikam,DC=com");
DirectorySearcher searcher = new DirectorySearcher(entry);
searcher.Filter = "(objectClass=*)";
uint rangeStep = 1000;
uint rangeLow = 0;
uint rangeHigh = rangeLow + (rangeStep - 1);
bool lastQuery = false;
bool quitLoop = false;
do
{
string attributeWithRange;
if(!lastQuery)
{
attributeWithRange = String.Format("member;range={0}-{1}", rangeLow, rangeHigh);
}
else
{
attributeWithRange = String.Format("member;range={0}-*", rangeLow);
}
searcher.PropertiesToLoad.Clear();
searcher.PropertiesToLoad.Add(attributeWithRange);
SearchResult results = searcher.FindOne();
foreach(string res in results.Properties.PropertyNames)
{
System.Diagnostics.Debug.WriteLine(res.ToString());
}
if(results.Properties.Contains(attributeWithRange))
{
foreach(object obj in results.Properties[attributeWithRange])
{
Console.WriteLine(obj.GetType());
if(obj.GetType().Equals(typeof(System.String)))
{
}
else if (obj.GetType().Equals(typeof(System.Int32)))
{
}
Console.WriteLine(obj.ToString());
}
if(lastQuery)
{
quitLoop = true;
}
}
else
{
lastQuery = true;
}
if(!lastQuery)
{
rangeLow = rangeHigh + 1;
rangeHigh = rangeLow + (rangeStep - 1);
}
}
while(!quitLoop);
}
catch(Exception ex)
{
// Handle exception ex.
}
Source: http://systemmanager.ru/adam-sdk.en/netds/enumerating_members_in_a_large_group.htm

Related

Spring ldap unlocking an account

I am trying to unlock user account using spring ldap and getting the error message
""Malformed 'LockoutTime' attribute value" exception.
My code looks like below
public boolean unlockAccount(Name dn) {
ModificationItem item = new ModificationItem(DirContext.REPLACE_ATTRIBUTE, new BasicAttribute("lockoutTime", 0));
ldapTemplate.modifyAttributes(dn, new ModificationItem[] {item});
return true;
}
I am using Windows server 2016 and Spring ldap 2.3.2.
Is 'lockoutTime' the correct attribute to unlock an account ?
Is there anything else I am missing ?
In LDAP if you type the wrong password for more than 5 times, the account gets locked. If you want to unlock the user you have to delete an operational attribute name as pwdAccountLockedTime.
public String unlockUser(Users pvo) {
System.out.println("this is pvo" + pvo);
Name dn = buildDn(pvo);
DirContextOperations context = ldapTemplate.lookupContext(dn);
ModificationItem[] modificationItems;
modificationItems = new ModificationItem[1];
modificationItems[0] = new ModificationItem(DirContext.REMOVE_ATTRIBUTE,
new BasicAttribute("pwdAccountLockedTime"));
ldapTemplate.modifyAttributes(dn, modificationItems);
return "Account Unlocked";
}
build Dn for your LDAP and use the above code then the user gets unlocked.
String[] attrIDs = new String[] { "lockoutTime", "sAMAccountName",
"distinguishedName","pwdLastSet", "accountExpires", "userAccountControl",
"IsAccountLocked" };
ctls.setReturningAttributes(attrIDs);
ctls.setSearchScope(2);
String filter = "(&(objectClass=user)(objectCategory=Person)(sAMAccountName=" +
samaccountname+ "))";
NamingEnumeration<SearchResult> answer = ctx.search(adManagedOU, filter,ctls);
while (answer.hasMore()) {
SearchResult rs = answer.next();
Attributes attrs = rs.getAttributes();
distinguishedName = rs.getNameInNamespace();
String[] lockouttime = null;
String lockOutValue=attrs.get("lockoutTime");
if (lockOutValue != null)
{
lockouttime = attrs.get("lockoutTime").toString().split(":");
if (Long.valueOf(lockouttime[1].trim()) > 0) {
ModificationItem[] mods1 = new ModificationItem[] {
new ModificationItem(2, new BasicAttribute("lockoutTime", "0") )
};
((DirContext) ctls).modifyAttributes(distinguishedName, mods1);
}
else
{
LOGGER.info(username + " Account Not Locked");
}
The only values that may be set on lockouttime is to set the value to "0" which will effectively un-lock the account.
To learn more on Microsoft Active Directory Lockouts.
Setting the value to a String instead of an int makes this work, at least with AWS Simple AD.
ModificationItem item = new ModificationItem(DirContext.REPLACE_ATTRIBUTE, new BasicAttribute("lockoutTime", "0"));
ldapTemplate.modifyAttributes(dn, new ModificationItem[] {item});

Export data from excel to a database table using ClosedXML example

i have to read rows of data from an excel file which has only a column and then i have to save the rows in a table in database .
In my project i have to use ClosedXML .dll .
I have search but i couldn't find an example .
Can you please help me?
Thanks
For the ClosedXML part, you can refer to the documentation at https://github.com/ClosedXML/ClosedXML/wiki/Finding-and-extracting-the-data
private static void Main()
{
List<String> categories;
List<String> companies;
ExtractCategoriesCompanies("NorthwindData.xlsx", out categories, out companies);
// Do something with the categories and companies
}
private static void ExtractCategoriesCompanies(string northwinddataXlsx, out List<string> categories, out List<string> companies)
{
categories = new List<string>();
const int coCategoryId = 1;
const int coCategoryName = 2;
var wb = new XLWorkbook(northwinddataXlsx);
var ws = wb.Worksheet("Data");
// Look for the first row used
var firstRowUsed = ws.FirstRowUsed();
// Narrow down the row so that it only includes the used part
var categoryRow = firstRowUsed.RowUsed();
// Move to the next row (it now has the titles)
categoryRow = categoryRow.RowBelow();
// Get all categories
while (!categoryRow.Cell(coCategoryId).IsEmpty())
{
String categoryName = categoryRow.Cell(coCategoryName).GetString();
categories.Add(categoryName);
categoryRow = categoryRow.RowBelow();
}
// There are many ways to get the company table.
// Here we're using a straightforward method.
// Another way would be to find the first row in the company table
// by looping while row.IsEmpty()
// First possible address of the company table:
var firstPossibleAddress = ws.Row(categoryRow.RowNumber()).FirstCell().Address;
// Last possible address of the company table:
var lastPossibleAddress = ws.LastCellUsed().Address;
// Get a range with the remainder of the worksheet data (the range used)
var companyRange = ws.Range(firstPossibleAddress, lastPossibleAddress).RangeUsed();
// Treat the range as a table (to be able to use the column names)
var companyTable = companyRange.AsTable();
// Get the list of company names
companies = companyTable.DataRange.Rows()
.Select(companyRow => companyRow.Field("Company Name").GetString())
.ToList();
}
EDIT: The below only gets you a populated datatable. You'll then need to load that datatable into your database. You don't say which database this is, but for SQL Server you would use the SqlBulkCopy class (see definition, which also has an example).
Late to the party, but try this:
public static DataTable GetDataTableFromExcel(string path, string sheetname = "", bool hasHeader = true)
{
using (var workbook = new XLWorkbook(path))
{
IXLWorksheet worksheet;
if (string.IsNullOrEmpty(sheetname))
worksheet = workbook.Worksheets.First();
else
worksheet = workbook.Worksheets.FirstOrDefault(x => x.Name == sheetname);
var rangeRowFirst = worksheet.FirstRowUsed().RowNumber();
var rangeRowLast = worksheet.LastRowUsed().RowNumber();
var rangeColFirst = worksheet.FirstColumnUsed().ColumnNumber();
var rangeColLast = worksheet.LastColumnUsed().ColumnNumber();
DataTable tbl = new DataTable();
for (int col = rangeColFirst; col <= rangeColLast; col++)
tbl.Columns.Add(hasHeader ? worksheet.FirstRowUsed().Cell(col).Value.ToString() : $"Column {col}");
Logger("started creating datatable");
rangeRowFirst = rangeRowFirst + (hasHeader ? 1 : 0);
var colCount = rangeColLast - rangeColFirst;
for (int rowNum = rangeRowFirst; rowNum <= rangeRowLast; rowNum++)
{
List<string> colValues = new List<string>();
for (int col = 1; col <= colCount; col++)
{
colValues.Add(worksheet.Row(rowNum).Cell(col).Value.ToString());
}
tbl.Rows.Add(colValues.ToArray());
}
Logger("finished creating datatable");
return tbl;
}
}
and call like this:
var datatable = GetDataTableFromExcel(fileName, sheetName);
If you're using (the excellent and free in its basic form) LinqPad, you can inspect the datatable using datatable.Dump();

Fastest way to record all DocIds and FileNames from dtSearch in SQL database

I am using dtSearch on combination with a SQL database and would like to maintain a table that includes all DocIds and their related FileNames. From there, I will add a column with my foreign key to allow me to combine text and database searches.
I have code to simply return all the records in the index and add them one by one to the DB. This, however, takes FOREVER, and doesn't address the issue of how to simply append new records as they are added to the index. But just in case it helps:
MyDatabaseContext db = new StateScapeEntities();
IndexJob ij = new dtSearch.Engine.IndexJob();
ij.IndexPath = #"d:\myindex";
IndexInfo indexInfo = dtSearch.Engine.IndexJob.GetIndexInfo(#"d:\myindex");
bool jobDone = ij.Execute();
SearchResults sr = new SearchResults();
uint n = indexInfo.DocCount;
for (int i = 1; i <= n; i++)
{
sr.AddDoc(ij.IndexPath, i, null);
}
for (int i = 1; i <= n; i++)
{
sr.GetNthDoc(i - 1);
//IndexDocument is defined elsewhere
IndexDocument id = new IndexDocument();
id.DocId = sr.CurrentItem.DocId;
id.FilePath = sr.CurrentItem.Filename;
if (id.FilePath != null)
{
db.IndexDocuments.Add(id);
db.SaveChanges();
}
}
To keep the DocId in the index you must use the flag dtsIndexKeepExistingDocIds in the IndexJob
You can also look the dtSearch Text Retrieval Engine Programmer's Reference when the DocID is changed
When a document is added to an index, it is assigned a DocId, and DocIds are always numbered sequentially.
When a document is reindexed, the old DocId is cancelled and a new DocId is assigned.
When an index is compressed, all DocIds in the index are renumbered to remove the cancelled DocIds unless the dtsIndexKeepExistingDocIds flag is set in IndexJob.
When an index is merged into another index, DocIds in the target index are never changed. The documents merged into the target index will all be assigned new, sequentially-numbered DocIds, unless (a) the dtsIndexKeepExistingDocIds flag is set in IndexJob and (b) the indexes have non-overlapping ranges of doc ids.
To improve your speed you can search for the word "xfirstword" and get all documents in an index.
You can also look to the faq How to retrieve all documents in an index
So, I used part of user2172986's response, but combined it with some additional code to get the solution to my question. I did indeed have to set the dtsKeepExistingDocIds flag in my index update routine.
From there, I only wanted to add the newly created DocIds to my SQL database. For that, I used the following code:
string indexPath = #"d:\myindex";
using (IndexJob ij = new dtSearch.Engine.IndexJob())
{
//make sure the updated index doesn't change DocIds
ij.IndexingFlags = IndexingFlags.dtsIndexKeepExistingDocIds;
ij.IndexPath = indexPath;
ij.ActionAdd = true;
ij.FoldersToIndex.Add( indexPath + "<+>");
ij.IncludeFilters.Add( "*");
bool jobDone = ij.Execute();
}
//create a DataTable to hold results
DataTable newIndexDoc = MakeTempIndexDocTable(); //this is a custom method not included in this example; just creates a DataTable with the appropriate columns
//connect to the DB;
MyDataBase db = new MyDataBase(); //again, custom code not included - link to EntityFramework entity
//get the last DocId in the DB?
int lastDbDocId = db.IndexDocuments.OrderByDescending(i => i.DocId).FirstOrDefault().DocId;
//get the last DocId in the Index
IndexInfo indexInfo = dtSearch.Engine.IndexJob.GetIndexInfo(indexPath);
uint latestIndexDocId = indexInfo.LastDocId;
//create a searchFilter
dtSearch.Engine.SearchFilter sf = new SearchFilter();
int indexId = sf.AddIndex(indexPath);
//only select new records (from one greater than the last DocId in the DB to the last DocId in the index itself
sf.SelectItems(indexId, lastDbDocId + 1, int.Parse(latestIndexDocId.ToString()), true);
using (SearchJob sj = new dtSearch.Engine.SearchJob())
{
sj.SetFilter(sf);
//return every document in the specified range (using xfirstword)
sj.Request = "xfirstword";
// Specify the path to the index to search here
sj.IndexesToSearch.Add(indexPath);
//additional flags and limits redacted for clarity
sj.Execute();
// Store the error message in the status
//redacted for clarity
SearchResults results = sj.Results;
int startIdx = 0;
int endIdx = results.Count;
if (startIdx==endIdx)
return;
for (int i = startIdx; i < endIdx; i++)
{
results.GetNthDoc(i);
IndexDocument id = new IndexDocument();
id.DocId = results.CurrentItem.DocId;
id.FileName= results.CurrentItem.Filename;
if (id.FileName!= null)
{
DataRow row = newIndexDoc.NewRow();
row["DocId"] = id.DocId;
row["FileName"] = id.FileName;
newIndexDoc.Rows.Add(row);
}
}
newIndexDoc.AcceptChanges();
//SqlBulkCopy
using (SqlConnection connection =
new SqlConnection(db.Database.Connection.ConnectionString))
{
connection.Open();
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection))
{
bulkCopy.DestinationTableName =
"dbo.IndexDocument";
try
{
// Write from the source to the destination.
bulkCopy.WriteToServer(newIndexDoc);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
newIndexDoc.Clear();
db.UpdateIndexDocument();
}
Here is my new solution with AddDoc method from the SearchResults interface:
First get the StartingDocID and the LastDocID from the IndexInfo and walk the loop like this:
function GetFilename(paDocID: Integer): String;
var
lCOMSearchResults: ISearchResults;
lSearchResults_Count: Integer;
begin
if Assigned(prCOMServer) then
begin
lCOMSearchResults := prCOMServer.NewSearchResults as ISearchResults;
lCOMSearchResults.AddDoc(GetIndexPath(prIndexContent), paDocID, 0);
lSearchResults_Count := lCOMSearchResults.Count;
if lSearchResults_Count = 1 then
begin
lCOMSearchResults.GetNthDoc(0);
Result := lCOMSearchResults.DocDetailItem['_Filename'];
end;
end;
end

Salesforce Add value in 2d array (Index out of bound 0)

I am trying to add array in an array i.e 2d array, while I am trying it throws this error:
List index out of bound 0
I am trying at this line:
df_depts_obs[df_mk_index] = departmentobs_columns;
private void initializeInconsistentRecommendations() {
recmn_qty = new List<Integer>();
String mk_unique_Data;
Integer bookmark;
bookmark=0;
mk_unique_Data='';
totalCount = 0;
inconsistentRecommendations = new List<AA_Recommendations__c>();
inconsistentAccessorys = new List<AA_Accessory__c>();
df_depts = new List<AA_DepartmentObservation__c>();
df_depts_obs = new List<List<AA_DepartmentObservation__c>>();
df_depts_recmds = new List<List<AA_DepartmentRecommendation__c>>();
df_depts_prds = new List<List<AA_Recommendations__c>>();
String df_mk_unique_dept='';
Integer df_mk_index=-1;
departmentRecommendations_data = new List<AA_DepartmentRecommendation__c>();
List<AA_DepartmentObservation__c>departmentObservations = [SELECT Id, Name, Observation_ID__c,Department_ID__r.Name,Observation_ID__r.Observation__c FROM AA_DepartmentObservation__c
WHERE Department_ID__r.Assessment__c = :assessment.Id LIMIT 10];
if (departmentObservations.size() > 0) {
for (integer index = 0; index < departmentObservations.size(); index++ ) {
AA_DepartmentObservation__c departmentObservation = departmentObservations[index];
//df_depts.add(departmentObservation); // FOR DEPARTMNET HEADING
if(!df_mk_unique_dept.contains((String)departmentObservations[index].Department_ID__r.Name)){
df_mk_unique_dept += '<li>'+departmentObservations[index].Department_ID__r.Name+'</li>';
df_depts.add(departmentObservation); // FOR DEPARTMNET HEADING
df_mk_index = df_mk_index + 1;
}
List<AA_DepartmentObservation__c> departmentobs_columns = new List<AA_DepartmentObservation__c>();
departmentobs_columns.add(departmentObservation);
df_depts_obs[df_mk_index] = departmentobs_columns;
In Apex, you cannot use array notation [ ] to add objects, only set values for indexes in the array already exists.
You can either declare the size of the array so that it is initialized, or grow the array dynamically using the add method.
This example shows both options:
List<List<String>> stringList = new List<List<String>>(1);
List<String> innerListOne = new List<String> {'Red','Blue'};
List<String> innerListTwo = new List<String> {'One','Two'};
stringList[0] = innerListOne;
stringList.add(innerListTwo);

What is the meaning and purpose of Active Directory Primary Group?

I recently heard of the Primary Group concept in Active Directory but it so far it's quite unclear that is its purpose and how this really works.
As part of the same question, how do I get the Primary Group using LDAP ? It seems that this is not listed as a memberOf attribute.
I've been using this code to get the primary group of a user in Active Directory:
private string GetPrimaryGroup(DirectoryEntry aEntry, DirectoryEntry aDomainEntry)
{
int primaryGroupID = (int)aEntry.Properties["primaryGroupID"].Value;
byte[] objectSid = (byte[])aEntry.Properties["objectSid"].Value;
StringBuilder escapedGroupSid = new StringBuilder();
// Copy over everything but the last four bytes(sub-authority)
// Doing so gives us the RID of the domain
for(uint i = 0; i < objectSid.Length - 4; i++)
{
escapedGroupSid.AppendFormat("\\{0:x2}", objectSid[i]);
}
// Add the primaryGroupID to the escape string to build the SID of the primaryGroup
for(uint i = 0; i < 4; i++)
{
escapedGroupSid.AppendFormat("\\{0:x2}", (primaryGroupID & 0xFF));
primaryGroupID >>= 8;
}
// Search the directory for a group with this SID
DirectorySearcher searcher = new DirectorySearcher();
if(aDomainEntry != null)
{
searcher.SearchRoot = aDomainEntry;
}
searcher.Filter = "(&(objectCategory=Group)(objectSID=" + escapedGroupSid.ToString() + "))";
searcher.PropertiesToLoad.Add("distinguishedName");
return searcher.FindOne().Properties["distinguishedName"][0].ToString();
}
The primary group is a group that every new user is added to by default - usually Domain Users.

Resources