I'm trying to get programmatically what I can get manually from SSMS using Tasks > Generate Scripts
The code below works fine, EXCEPT it doesn't generate any constraints. I don't get any ALTER TABLE [foo] ADD CONSTRAINT ... ON DELETE CASCADE etc etc. I've tried a lot of combinations of Dri options and on different databases as well. I'm stumped.
Thanks for insight!
Scripter scrp = new Scripter(srv)
{
Options =
{
ScriptDrops = false,
WithDependencies = false,
Indexes = true,
Triggers = false,
Default = true,
DriAll = true,
//ScriptData = true,
ScriptSchema = true,
}
};
var urns = new List<Urn>();
foreach (Table tb in db.Tables)
{
if (tb.IsSystemObject == false)
{
urns.Add(tb.Urn);
}
}
var inserts = scrp.EnumScript(urns.ToArray());
File.WriteAllLines(path, inserts);
Well, I found a solution, which is to use the Script method of each object to produce the schema and the EnumScript method (with scriptSchema=false) to produce the inserts for the table content.
foreach (Table tb in db.Tables)
{
if (tb.IsSystemObject == false)
{
foreach (var s in tb.Script(schemaOptions))
strings.Add(s);
if (scriptData)
{
foreach (var i in tb.EnumScript(insertOptions))
strings.Add(i);
}
}
}
I confess this solution feels a bit hollow because I never found out why the original method didn't work. It's a Repair without a Diagnosis, but a repair nonetheless.
As to why I wrote this thing in the first place, my database is on a shared server and there isn't any way to get an automated backup that I could use offline or somewhere else. So this is my backup scheme.
The solution above follows the code example given by Microsoft here: Scripting . The problem with this approach is the tables are scripted in No Particular Order, but need to be in order of their dependencies in order for the constraints to be defined and for rows to be inserted. Can't reference a foreign key in a table that doesn't exist yet.
The best solution I have so far is to use DependencyWalker.DiscoverDependencies() to get a
dependency tree, DependencyWalker.WalkDependencies() to get a linear list and iterate over that list, as follows:
var urns = new List<Urn>();
Scripter schemaScripter = new Scripter(srv) { Options = schemaOptions };
Scripter insertScripter = new Scripter(srv) { Options = insertOptions };
var dw = new DependencyWalker(srv);
foreach (Table t in db.Tables)
if (t.IsSystemObject == false)
urns.Add(t.Urn);
DependencyTree dTree = dw.DiscoverDependencies(urns.ToArray(), true);
DependencyCollection dColl = dw.WalkDependencies(dTree);
foreach (var d in dColl)
{
foreach (var s in schemaScripter.Script(new Urn[] { d.Urn }))
strings.Add(s);
strings.Add("GO");
if (scriptData)
{
int n = 0;
foreach (var i in insertScripter.EnumScript(new Urn[] {d.Urn}))
{
strings.Add(i);
if ((++n) % 100 == 0)
strings.Add("GO");
}
}
}
...
File.WriteAllLines(path, strings);
Adding a "GO" every so often keeps the batch size small so SSMS doesn't run out of memory.
To complete the example, the database gets scripted thus:
foreach (var s in db.Script(new ScriptingOptions { ScriptSchema = true }))
strings.Add(s);
strings.Add("GO");
strings.Add("use " + dbName);
strings.Add("GO");
Users, views, stored procedures are scripted thus:
foreach (User u in db.Users)
{
if (u.IsSystemObject == false)
{
foreach (var s in u.Script(new ScriptingOptions { ScriptSchema = true }))
strings.Add(s);
}
}
The file produced by this code can be used to recreate the database. I have it set up on an old laptop to pull a snapshot of my online database every hour. Poor man's log shipping / backups / mirroring.
Related
I have my database table named 'JobInfos' in SQL Server which contains many columns.
JobID - (int) auto populates incrementing value when data added
OrgCode - (string)
OrderNumber - (int)
WorkOrder - (int)
Customer - (string)
BaseModelItem - (string)
OrdQty - (int)
PromiseDate - (string)
LineType -(string)
This table gets written to many times a day using a Blazor application with Entity Framework and CSVHelper. This works perfectly. All rows from the CSV file are added to the database.
if (fileExist)
{
using (var reader = new StreamReader(#path))
using (var csv = new CsvReader(reader, config))
{
var records = csv.GetRecords<CsvRow>().Select(row => new JobInfo()
{
OrgCode = row.OrgCode,
OrderNumber = row.OrderNumber,
WorkOrder = row.WorkOrder,
Customer = row.Customer,
BaseModelItem = row.BaseModelItem,
OrdQty = row.OrdQty,
PromiseDate = row.PromiseDate,
LineType = row.LineType,
});
using (var db = new ApplicationDbContext())
{
while (!reader.EndOfStream)
{
if (lineNumber != 0)
{
db.AddRange(records.ToList());
db.SaveChanges();
}
lineNumber++;
}
NavigationManager.NavigateTo("/", true);
}
}
As these multiple CSV files can contain rows that may already be in the database table, I am getting duplicate records when the table is read from, which causes the users to delete all the newer duplicate rows manually to only keep the original entry.
I have no control over the CSV files or their creation. I am trying to only add rows that contain new data based on the WorkOrder number which can not be the same as any others.
I found another post here on StackOverflow which helps but I am stuck with a remaining error I can't figure out.
The Helpful post
I changed my code here...
if (lineNumber != 0)
{
var recordworkorder = records.Select(x => x.WorkOrder).ToList();
var workordersindb = db.JobInfos.Where(x => recordworkorder.Contains(x.WorkOrder)).ToList();
var workordersNotindb = records.Where(x => !workordersindb.Contains(x.WorkOrder));
db.AddRange(records.ToList(workordersNotindb));
db.SaveChanges();
}
but this line...
var workordersNotindb = records.Where(x => !workordersindb.Contains(x.WorkOrder));`
throws an error at the end (x.WorkOrder) - CS1503 Argument 1: cannot convert from 'int' to 'DepotQ4.Data.JobInfo'
WorkOrder is an int
JobID is the Primary Key and an int
Every record in the table must have a unique WorkOrder
I am not sure what I am not seeing. Could use some help here please?
Your variable workordersindb is a List<JobInfo>. So when you try to select from records.Where(x => !workordersindb.Contains(x.WorkOrder)) you are trying to match the list of JobInfo in workordersindb to the int of x.WorkOrder. workordersindb needs to be a List<int> in order to be able to use it with the Contains. records would have had the same issue, but you solved it by creating the variable recordworkorder and using records.Select(x => x.WorkOrder) to get a List<int>.
if (lineNumber != 0)
{
var recordworkorder = records.Select(x => x.WorkOrder).ToList();
var workordersindb = db.JobInfos.Where(x => recordworkorder.Contains(x.WorkOrder)).Select(x => x.WorkOrder).ToList();
var workordersNotindb = records.Where(x => !workordersindb.Contains(x.WorkOrder));
db.JobInfos.AddRange(workordersNotindb);
db.SaveChanges();
}
I have this function and it is working perfectly
public DemandeConge Creat(DemandeConge DemandeConge)
{
try
{
var _db = Context;
int numero = 0;
//??CompanyStatique
var session = _httpContextAccessor.HttpContext.User.Claims.ToList();
int currentCompanyId = int.Parse(session[2].Value);
numero = _db.DemandeConge.AsEnumerable()
.Where(t => t.companyID == currentCompanyId)
.Select(p => Convert.ToInt32(p.NumeroDemande))
.DefaultIfEmpty(0)
.Max();
numero++;
DemandeConge.NumeroDemande = numero.ToString();
//_db.Entry(DemandeConge).State = EntityState.Added;
_db.DemandeConge.Add(DemandeConge);
_db.SaveChanges();
return DemandeConge;
}
catch (Exception e)
{
return null;
}
}
But just when i try to insert another leave demand directly after inserting one (without waiting or refreshing the page )
An error appears saying that this new demand.id exists
I think that i need to add refresh after saving changes?
Any help and thanks
Code like this:
numero = _db.DemandeConge.AsEnumerable()
.Where(t => t.companyID == currentCompanyId)
.Select(p => Convert.ToInt32(p.NumeroDemande))
.DefaultIfEmpty(0)
.Max();
numero++;
Is a very poor pattern. You should leave the generation of your "numero" (ID) up to the database via an Identity column. Set this up in your DB (if DB First) and set up your mapping for this column as DatabaseGenerated.Identity.
However, your code raises lots of questions.. Why is it a String instead of an Int? This will be a bugbear for using an identity column.
The reason you will want to avoid code like this is because each request will want to query the database to get the "max" ID, as soon as you get two requests running relatively simultaneously you will get 2 requests that say the max ID is "100" before either can reserve and insert 101, so both try to insert 101. By using Identity columns the database will get 2x inserts and give them an ID first-come-first-serve. EF can manage associating FKs around these new IDs automatically for you when you set up navigation properties for the relations. (Rather than trying to set FKs manually which is the typical culprit for developers trying to fetch a new ID app-side)
If you're stuck using an existing schema where the PK is a combination of company ID and this Numero column as a string then about all you can do is implement a retry strategy to account for duplicates:
const int MAXRETRIES = 5;
var session = _httpContextAccessor.HttpContext.User.Claims.ToList();
int currentCompanyId = int.Parse(session[2].Value);
int insertAttemptCount = 0;
while(insertAttempt < MAXRETRIES)
{
try
{
numero = Context.DemandeConge
.Where(t => t.companyID == currentCompanyId)
.Select(p => Convert.ToInt32(p.NumeroDemande))
.DefaultIfEmpty(0)
.Max() + 1;
DemandeConge.NumeroDemande = numero.ToString();
Context.DemandeConge.Add(DemandeConge);
Context.SaveChanges();
break;
}
catch (UpdateException)
{
insertAttemptCount++;
if (insertAttemptCount >= MAXRETRIES)
throw; // Could not insert, throw and handle exception rather than return #null.
}
}
return DemandeConge;
Even this won't be fool proof and can result in failures under load, plus it is a lot of code to work around a poor DB design so my first recommendation would be to fix the schema because coding like this is prone to errors and brittle.
When Indexing the items, it fails sometimes and it gives,
The remote server returned an error: (504) Gateway Timeout. [The remote server returned an error: (504) Gateway Timeout.]
The Indexing logic is here as below,
var client = EPiServer.Find.Framework.SearchClient.Instance;
List<ItemModel> items = getItems(); // Get more than 1000 items
List<ItemModel> tempItems = new List<ItemModel>();
//Index 50 items at a time
foreach(var item in items)
{
tempItems.Add(item);
if (tempItems.Count == 50)
{
client.Index(tempItems);
tempItems.Clear();
}
}
What causes this to happen ?
Note: The above mentioned ItemModel is a custom model which is not implemented interfaces (such as IContent). And the items is a list of ItemModel objects.
Additional info:
EPiServer.Find.Framework version 13.0.1
EPiServer.CMS.Core version 11.9.2
I always figured the SearchClient to be a bit sketchy when manipulating data in Find, as far as I figured (but I have to check this) the SearchClient obey under the request limitation of Episerver Find and when doing bigger operations in loops it tends to time out.
Instead, use the ContentIndexer, i.e.
// Use this or injected parameter
var loader = ServiceLocator.Current.GetInstance<IContentLoader>();
// Remove all children or not
var cascade = true;
ContentReference entryPoint = ...where you want to start
// Get all indexable languages from Find
Languages languages = SearchClient.Instance.Settings.Languages;
// Remove all current instances of all languages below the selected content node
//languages.ForEach(x => ContentIndexer.Instance.RemoveFromIndex(entryPoint, cascade.Checked, x.FieldSuffix));
foreach (var lang in languages)
{
if (cascade)
{
var descendents = loader.GetDescendents(entryPoint);
foreach (ContentReference descendent in descendents)
{
ContentIndexer.Instance.RemoveFromIndex(descendent, false, lang.FieldSuffix);
}
}
// Try delete the entrypoint
var entryTest = loader.Get<IContent>(entryPoint, new CultureInfo(lang.FieldSuffix));
if (entryTest != null)
{
var delRes = ContentIndexer.Instance.Delete(entryTest);
}
}
This is the most bulletproof way to delete stuff from the index as far as I figured.
I have a table that consists of a column of pre-populated numbers. My API using Nhibernate grabs the first 10 rows where 'Used' flag is set as false.
What would be the best possible way to avoid concurrency issue when multiple session try to grab row from the table?
After selecting the row, I can update the flag column to be True so subsequent calls will not use the same numbers.
With such a general context, it could be done that way:
// RepeatableRead ensures the read rows does not get concurrently updated by another
// session.
using (var tran = session.BeginTransaction(IsolationLevel.RepeatableRead))
{
var entities = session.Query<Entity>()
.Where(e => !e.Used)
.OrderBy(e => e.Id)
.Take(10)
.ToList();
foreach(var entity in entities)
{
e.Used = true;
}
// If your session flush mode is not the default one and does not cause
// commits to flush the session, add a session.Flush(); call before committing.
tran.Commit();
return entities;
}
It is simple. It may fail with a deadlock, in which case you would have to throw away the session, get a new one, and retry.
Using an optimistic update pattern could be an alternate solution, but this requires some code for recovering from failed attempts too.
Using a no explicit lock solution, which will not cause deadlock risks, could do it, but it will require more queries:
const int entitiesToObtain = 10;
// Could initialize here with null instead, but then, will have to check
// for null after the while too.
var obtainedEntities = new List<Entity>();
while (obtainedEntities.Count == 0)
{
List<Entity> candidates;
using (var tran = session.BeginTransaction())
{
candidatesIds = session.Query<Entity>()
.Where(e => !e.Used)
.Select(e => e.Id)
.OrderBy(id => id)
.Take(entitiesToObtain)
.ToArray();
}
if (candidatesIds.Count == 0)
// No available entities.
break;
using (var tran = session.BeginTransaction())
{
var updatedCount = session.CreateQuery(
#"update Entity e set e.Used = true
where e.Used = false
and e.Id in (:ids)")
.SetParameterList("ids", candidatesIds)
.ExecuteUpdate();
if (updatedCount == candidatesIds.Length)
{
// All good, get them.
obtainedEntities = session.Query<Entity>()
.Where(e => candidatesIds.Contains(e.Id))
.ToList();
tran.Commit();
}
else
{
// Some or all of them were no more available, and there
// are no reliable way to know which ones, so just try again.
tran.Rollback();
}
}
}
This uses NHibernate DML-style operations as suggested here. A strongly typed alternative is available in NHibernate v5.0.
I am trying to determine a way to audit which records a given user can see by;
Object Type
Record Type
Count of records
Ideally would also be able to see which fields for each object/record type the user can see.
We will need to repeat this often and for different users and in different orgs, so would like to avoid manually determining this.
My first thought was to create an app using the partner WSDL, but would like to ask if there are any easier approaches or perhaps existing solutions.
Thanks all
I think that you can follow the documentation to solve it, using a query similar to this one:
SELECT RecordId
FROM UserRecordAccess
WHERE UserId = [single ID]
AND RecordId = [single ID] //or Record IN [list of IDs]
AND HasReadAccess = true
The following query returns the records for which a queried user has
read access to.
In addition, you should add limit 1 and get from record metadata the object type,record type, and so on.
I ended up using the below (C# using the Partner WSDL) to get an idea of what kinds of objects the user had visibility into.
Just a quick'n'dirty utility for my own use (read - not prod code);
var service = new SforceService();
var result = service.login("UserName", "Password");
service.Url = result.serverUrl;
service.SessionHeaderValue = new SessionHeader { sessionId = result.sessionId };
var queryResult = service.describeGlobal();
int total = queryResult.sobjects.Count();
int batcheSize = 100;
var batches = Math.Ceiling(total / (double)batcheSize);
using (var output = new StreamWriter(#"C:\test\sfdcAccess.txt", false))
{
for (int batch = 0; batch < batches; batch++)
{
var toQuery =
queryResult.sobjects.Skip(batch * batcheSize).Take(batcheSize).Select(x => x.name).ToArray();
var batchResult = service.describeSObjects(toQuery);
foreach (var x in batchResult)
{
if (!x.queryable)
{
Console.WriteLine("{0} is not queryable", x.name);
continue;
}
var test = service.query(string.Format("SELECT Id FROM {0} limit 100", x.name));
if(test == null || test.records == null)
{
Console.WriteLine("{0}:null records", x.name);
continue;
}
foreach (var record in test.records)
{
output.WriteLine("{0}\t{1}",x.name, record.Id);
}
Console.WriteLine("{0}:\t{1} records(0)", x.name, test.size);
}
}
output.Flush();
}