Akka.Net PersistenceQuery not returning all results - akka-stream

I am using Akka.Net (v 1.3.2) and am trying to query the event journal for all events with a specific tag. I only want the events that exist at the time I query the journal. Inside an actor, I have the following code:
var readJournal = PersistenceQuery.Get(Context.System).ReadJournalFor<SqlReadJournal>(SqlReadJournal.Identifier);
var stream = readJournal.CurrentEventsByTag("The Tag Name", Offset.NoOffset());
var materializer = ActorMaterializer.Create(Context.System);
stream.RunForeach(envelope =>
{
// Do some stuff with the EventEnvelope
}, materializer).Wait();
This will successfully query the event journal. However, the problem is it will only return the first 100 events. I need all of them that match the query!
Question: How do I remove the limit/filter that exists when querying the event journal by tag name?
If you need it, here is my akka.persistence configuration:
var config = Akka.Configuration.ConfigurationFactory.ParseString(#"
akka.persistence {
journal {
plugin = ""akka.persistence.journal.sql-server""
sql-server {
class = ""Akka.Persistence.SqlServer.Journal.SqlServerJournal, Akka.Persistence.SqlServer""
connection-string = """ + connectionString + #"""
schema-name = dbo
table-name = __akka_EventJournal
metadata-table-name = __akka_Metadata
auto-initialize = on
}
}
snapshot-store {
plugin = ""akka.persistence.snapshot-store.sql-server""
sql-server {
class = ""Akka.Persistence.SqlServer.Snapshot.SqlServerSnapshotStore, Akka.Persistence.SqlServer""
connection-string = """ + connectionString + #"""
schema-name = dbo
table-name = __akka_SnapshotStore
auto-initialize = on
}
}
}"
);

There are two things to check out:
You can set the maximum number of messages returned in one query by setting up akka.persistence.query.journal.sql.max-buffer-size value (see: reference.conf).
Use readJournal.EventsByTag instead of readJournal.CurrentEventsByTag to get a continuous stream of events. Just keep in mind, that it won't complete by itself, but will live on waiting for new events to arrive. You can stop it explicitly i.e. by using KillSwitch.

Related

Multiple entries on DbUpdateException when one is expected

I sync data from an api and detect if an insert or update is necessary.
From time to time I receive DbUpdateExceptions and then fallback to single insert/update + savechanges instead of addrange/updaterange + savechanges.
Because single entities are so slow I wanted to only remove the failing entity from changetracking and try to save it all again, but unfortunately mssql returns all entities instead of only the one that is failing in DbUpdateException.Entries.
Intellisense tells me
Gets the entries that were involved in the error. Typically this is a single entry, but in some cases it may be zero or multiple entries.
Interestingly this is true if I try it on a mysql server. There only one entity is returned, but mssql returns all, which makes it impossible for me to exclude only the failing one.
Is there any setting to change mssql behaviour?
Both mysql and mssql are azure hosted resources.
Here an example:
var addList = new List<MyEntity>();
var updateList = new List<MyEntity>();
//load existing data from db
var existingData = Context.Set<MyEntity>()
.AsNoTracking()
.Take(2).ToList();
if (existingData.Count < 2)
return;
//addList
addList.Add(new MyEntity
{
NotNullableProperty = "Value",
RequiredField1 = Guid.Empty,
RequiredField2 = Guid.Empty,
});
addList.Add(new MyEntity
{
NotNullableProperty = "Value",
RequiredField1 = Guid.Empty,
RequiredField2 = Guid.Empty,
});
addList.Add(existingData.ElementAt(0)); //this should fail due to duplicate key
addList.Add(new MyEntity
{
NotNullableProperty = "Value",
RequiredField1 = Guid.Empty,
RequiredField2 = Guid.Empty,
});
//updateList
existingData.ElementAt(1).NotNullableProperty = null; //this should fail due to invalid value
updateList.Add(existingData.ElementAt(1));
//save a new entity that should fail
var newKb = new MyEntity
{
NotNullableProperty = "Value",
RequiredField1 = Guid.Empty,
RequiredField2 = Guid.Empty,
};
Context.Add(newKb);
Context.SaveChanges();
newKb.NotNullableProperty = "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"; //this should fail due to length
updateList.Add(newKb);
try
{
if (addList.IsNotNullOrEmpty())
context.Set<MyEntity>().AddRange(addList);
if (updateList.IsNotNullOrEmpty())
context.Set<MyEntity>().UpdateRange(updateList);
context.SaveChanges();
}
catch (DbUpdateException updateException)
{
//updateException.Entries contains all entries, that were added/updated although only three should fail
}

Audit of what records a given user can see in SalesForce.com

I am trying to determine a way to audit which records a given user can see by;
Object Type
Record Type
Count of records
Ideally would also be able to see which fields for each object/record type the user can see.
We will need to repeat this often and for different users and in different orgs, so would like to avoid manually determining this.
My first thought was to create an app using the partner WSDL, but would like to ask if there are any easier approaches or perhaps existing solutions.
Thanks all
I think that you can follow the documentation to solve it, using a query similar to this one:
SELECT RecordId
FROM UserRecordAccess
WHERE UserId = [single ID]
AND RecordId = [single ID] //or Record IN [list of IDs]
AND HasReadAccess = true
The following query returns the records for which a queried user has
read access to.
In addition, you should add limit 1 and get from record metadata the object type,record type, and so on.
I ended up using the below (C# using the Partner WSDL) to get an idea of what kinds of objects the user had visibility into.
Just a quick'n'dirty utility for my own use (read - not prod code);
var service = new SforceService();
var result = service.login("UserName", "Password");
service.Url = result.serverUrl;
service.SessionHeaderValue = new SessionHeader { sessionId = result.sessionId };
var queryResult = service.describeGlobal();
int total = queryResult.sobjects.Count();
int batcheSize = 100;
var batches = Math.Ceiling(total / (double)batcheSize);
using (var output = new StreamWriter(#"C:\test\sfdcAccess.txt", false))
{
for (int batch = 0; batch < batches; batch++)
{
var toQuery =
queryResult.sobjects.Skip(batch * batcheSize).Take(batcheSize).Select(x => x.name).ToArray();
var batchResult = service.describeSObjects(toQuery);
foreach (var x in batchResult)
{
if (!x.queryable)
{
Console.WriteLine("{0} is not queryable", x.name);
continue;
}
var test = service.query(string.Format("SELECT Id FROM {0} limit 100", x.name));
if(test == null || test.records == null)
{
Console.WriteLine("{0}:null records", x.name);
continue;
}
foreach (var record in test.records)
{
output.WriteLine("{0}\t{1}",x.name, record.Id);
}
Console.WriteLine("{0}:\t{1} records(0)", x.name, test.size);
}
}
output.Flush();
}

Preforming Bulk data transactions with SalesForce using .Net C#

I am new to SalesForce (3 months).
Thus far I have been able to create an application in C# that I can use to preform Inserts and Updates to the SalesForce database. These transactions are one at a time.
No I have the need to preform large scale transactions. For example updating thousands of records at a time. Doing them one by one would quickly put us over our allotted API calls per 24 hour period.
I want to utilize the available bulk transactions process to cut down on the number of API calls. Thus far I have not had much luck coding this nor have I found any such documentation.
If anyone could either provide some generic examples or steer me to reliable documentation on the subject I would greatly appreciate it.
FYI, the data I need to use to do the updates and inserts comes from an IBM Unidata database sitting on an AIX machine. So direct web services communication is not realy possible. Getting the data from Unidata has been my headache. I have that worked out. Now the bulk api to SalesForce is my new headache.
Thanks in advance.
Jeff
You don't mention which API you're currently using, but using the soap partner or enterprise APIs you can write records to salesforce 200 at a time. (the create/update/upsert calls all take an array of SObjects).
Using the bulk API you can send data in chunks of thousands of rows at a time.
You can find the documentation for both sets of APIs here
The answers already given are a good start; however, are you sure you need to actually write a custom app that uses the bulk API? The salesforce data loader is a pretty robust tool, includes a command line interface, and can use either the "normal" or bulk data API's. Unless you are needing to do fancy logic as part of your insert/updates, or some sort of more real-time / on-demand loading, the data loader is going to be a better option than a custom app.
(this is the SOAP code though, not the Salesforce "Bulk API" ; careful not to confuse the two)
Mighy be below code provide clear insight on how to do bulk insertion.
/// Demonstrates how to create one or more Account records via the API
public void CreateAccountSample()
{
Account account1 = new Account();
Account account2 = new Account();
// Set some fields on the account1 object. Name field is not set
// so this record should fail as it is a required field.
account1.BillingCity = "Wichita";
account1.BillingCountry = "US";
account1.BillingState = "KA";
account1.BillingStreet = "4322 Haystack Boulevard";
account1.BillingPostalCode = "87901";
// Set some fields on the account2 object
account2.Name = "Golden Straw";
account2.BillingCity = "Oakland";
account2.BillingCountry = "US";
account2.BillingState = "CA";
account2.BillingStreet = "666 Raiders Boulevard";
account2.BillingPostalCode = "97502";
// Create an array of SObjects to hold the accounts
sObject[] accounts = new sObject[2];
// Add the accounts to the SObject array
accounts[0] = account1;
accounts[1] = account2;
// Invoke the create() call
try
{
SaveResult[] saveResults = binding.create(accounts);
// Handle the results
for (int i = 0; i < saveResults.Length; i++)
{
// Determine whether create() succeeded or had errors
if (saveResults[i].success)
{
// No errors, so retrieve the Id created for this record
Console.WriteLine("An Account was created with Id: {0}",
saveResults[i].id);
}
else
{
Console.WriteLine("Item {0} had an error updating", i);
// Handle the errors
foreach (Error error in saveResults[i].errors)
{
Console.WriteLine("Error code is: {0}",
error.statusCode.ToString());
Console.WriteLine("Error message: {0}", error.message);
}
}
}
}
catch (SoapException e)
{
Console.WriteLine(e.Code);
Console.WriteLine(e.Message);
}
}
Please find the small code which may help you to insert the data into salesforce objects using c# and WSDL APIs. I stuck to much to write code in c#. I assigned using direct index after spiting you can use your ways.
I split the column using | (pipe sign). You may change this and also <br>, \n, etc. (row and column breaking)
Means you can enter N rows which are in your HTML/text file. I wrote the program to add order by my designers who put the order on other website and fetch the data from e-commerce website and who has no interface for the salesforce to add/view the order records. I created one object for the same. and add following columns in the object.
Your suggestions are welcome.
private SforceService binding; // declare the salesforce servive using your access credential
try
{
string stroppid = "111111111111111111";
System.Net.HttpWebRequest fr;
Uri targetUri = new Uri("http://abc.xyz.com/test.html");
fr = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(targetUri);
if ((fr.GetResponse().ContentLength > 0))
{
System.IO.StreamReader str = new System.IO.StreamReader(fr.GetResponse().GetResponseStream());
string allrow = str.ReadToEnd();
string stringSeparators = "<br>";
string[] row1 = Regex.Split(allrow, stringSeparators);
CDI_Order_Data__c[] cord = new CDI_Order_Data__c[row1.Length - 1];
for (int i = 1; i < row1.Length-1; i++)
{
string colstr = row1[i].ToString();
string[] allcols = Regex.Split(colstr, "\\|");
cord[i] = new CDI_Order_Data__c(); // Very important to create object
cord[i].Opportunity_Job_Order__c = stroppid;
cord[i].jobid__c = stroppid;
cord[i].order__c = allcols[0].ToString();
cord[i].firstname__c = allcols[1].ToString();
cord[i].name__c = allcols[2].ToString();
DateTime dtDate = Convert.ToDateTime(allcols[3]);
cord[i].Date__c = new DateTime(Convert.ToInt32(dtDate.Year), Convert.ToInt32(dtDate.Month), Convert.ToInt32(dtDate.Day), 0, 0, 0); //sforcedate(allcols[3]); //XMLstringToDate(allcols[3]);
cord[i].clientpo__c = allcols[4].ToString();
cord[i].billaddr1__c = allcols[5].ToString();
cord[i].billaddr2__c = allcols[6].ToString();
cord[i].billcity__c = allcols[7].ToString();
cord[i].billstate__c = allcols[8].ToString();
cord[i].billzip__c = allcols[9].ToString();
cord[i].phone__c = allcols[10].ToString();
cord[i].fax__c = allcols[11].ToString();
cord[i].email__c = allcols[12].ToString();
cord[i].contact__c = allcols[13].ToString();
cord[i].lastname__c = allcols[15].ToString();
cord[i].Rep__c = allcols[16].ToString();
cord[i].sidemark__c = allcols[17].ToString();
cord[i].account__c = allcols[18].ToString();
cord[i].item__c = allcols[19].ToString();
cord[i].kmatid__c = allcols[20].ToString();
cord[i].qty__c = Convert.ToDouble(allcols[21]);
cord[i].Description__c = allcols[22].ToString();
cord[i].price__c = Convert.ToDouble(allcols[23]);
cord[i].installation__c = allcols[24].ToString();
cord[i].freight__c = allcols[25].ToString();
cord[i].discount__c = Convert.ToDouble(allcols[26]);
cord[i].salestax__c = Convert.ToDouble(allcols[27]);
cord[i].taxcode__c = allcols[28].ToString();
}
try {
SaveResult[] saveResults = binding.create(cord);
}
catch (Exception ce)
{
Response.Write("Buld order update errror" +ce.Message.ToString());
Response.End();
}
if (str != null) str.Close();
}

Finding New and Updated Pages in EpiServer

I have a requirement to display lists of newly-created and updated pages in our Episerver intranet - say the last ten of each. I've tried using FindPagesWithCriteria but this returns no results. Here's the code I've tried:
PageDataCollection recentPages;
PropertyCriteriaCollection criteria;
PropertyCriteria upperBound;
PropertyCriteria lowerBound;
criteria = new PropertyCriteriaCollection();
upperBound = new PropertyCriteria();
upperBound.Condition = CompareCondition.LessThan;
upperBound.Type = PropertyDataType.Date;
upperBound.Value = DateTime.Today.ToString();
upperBound.Name = "Created"; // Or Saved for updated pages
criteria.Add(upperBound);
lowerBound = new PropertyCriteria();
lowerBound.Condition = CompareCondition.GreaterThan;
lowerBound.Type = PropertyDataType.Date;
lowerBound.Value = DateTime.Today.AddDays(-7).ToString();
lowerBound.Name = "Created";
criteria.Add(lowerBound);
recentPages = DataFactory.Instance.FindPagesWithCriteria(PageReference.StartPage, criteria);
I've also tried using the RecentlyChangedPagesFinder (as detailed here) - this returns some results, but when I try to use the set of results to build a PageCollection to databind into a PageList, again I get nothing output. And I can't see that I could use that for new pages, only updated ones.
The property name should be "PageCreated".
http://epiwiki.se/developing/properties/all-built-in-properties
You can also improve your FindPagesWithCriteria-syntax by going something like this:
var criterias = new PropertyCriteriaCollection
{
new PropertyCriteria()
{
Name = "SomeProp",
Type = PropertyDataType.PageType,
Value = "eh",
Condition = CompareCondition.Equal,
Required = true
},
new PropertyCriteria()
{
...
};
var pages = DataFactory.Instance.FindPagesWithCriteria(somePageLink, criterias);

SqlCacheDependecy command notification not working

I been trying to get sqlcachedependecy working, but it doesn't appear to work
I got the proper settings in my web.config and also global.asa, however when I run this query and the changes are made to the database from either with in or outside the web site the cached objects are not updated please someone help? I know its not because this query is querying a view, because I tested this using straight SqlDependecy and the notification works fine.
public IQueryable<VictoryList> GetVictoryList()
{
string cacheKey = HttpContext.Current.User.Identity.Name + "victoryCacheKey";
IQueryable<VictoryList> cachednews = (IQueryable<VictoryList>)HttpContext.Current.Cache.Get(cacheKey);
if (cachednews == null)
{
var results = from v in _datacontext.ViewVictoryLists
orderby _datacontext.GetNewId()
select new VictoryList
{
MemberID = v.MemberID,
Username = v.Aspnetusername,
Location = v.Location,
DaimokuGoal = v.DaimokuGoal,
PreviewImageID = v.PreviewImageID,
TotalDaimoku = v.TotalDaimoku,
TotalDeterminations = v.TotalDeterminations,
DeterminationID = v.DeterminationID,
DeterminationName = v.DeterminationName
};
results = results.ToList().AsQueryable();
SqlCacheDependencyAdmin.EnableNotifications(_datacontext.Connection.ConnectionString);
SqlCacheDependency dependency =
new SqlCacheDependency(_datacontext.GetCommand(results) as SqlCommand);
HttpContext.Current.Cache.Insert(cacheKey, results, dependency);
return results;
}
return cachednews;
}
According to the stated Limitations for creating a query for notification, listed at msdn...
The statement must not reference a view.

Resources