In this Gmail api documentation we clearly see the following statements:
History results are returned in chronological order (increasing historyId).
History IDs increase chronologically but are not contiguous with random gaps in between valid IDs.
I did a message.list call on a large mailbox and I am seeing this to not be the case.
Message ID
Historyid
internaldate
17fddf08be628452
29216912
3/31/2022 3:06:40 AM
17fddf08a5b47746
29260741
3/31/2022 3:06:38 AM
17fddf08a1233c88
29254708
3/31/2022 3:06:38 AM
17fddf08470d6012
29134895
3/30/2022 11:32:50 PM
17fddf082c135739
29216911
3/31/2022 3:06:32 AM
17fddf08278d8193
29079901
3/31/2022 12:31:24 AM
17fddf07f1db6d26
29216910
3/31/2022 3:06:36 AM
17fddf07d1d4a8da
29134894
3/31/2022 3:06:33 AM
17fddf07c5e32f18
29079900
3/30/2022 11:33:01 PM
17fddf07bf4cfa19
29254707
3/30/2022 11:33:07 PM
17fddf07bf3503c3
29260733
3/31/2022 3:06:35 AM
17fddf07b05d64b8
29079899
3/31/2022 3:06:36 AM
I can clearly see that the historyID is not in increasing order. I searched this issue and ran upon this issue tracker where the google representative claimed:
"The first message coming from the user.message.list method is in descending order of date but not of historyId."
Which is also not the case as we can see that some dates are also not in order.
Is there any consistency here or any way to get the results of message.list in some type of order?
public static IList<Message> ListUserMessages(ref string nextPageToken, int pageSize)
{
UsersResource.MessagesResource.ListRequest request = GmailService.Users.Messages.List("me");
request.IncludeSpamTrash = false;
request.MaxResults = pageSize;
if (!string.IsNullOrWhiteSpace(nextPageToken))
request.PageToken = nextPageToken;
ListMessagesResponse response = null;
IList<Message> messages = null;
response = request.Execute();
messages = response.Messages;
if (!string.IsNullOrEmpty(response.NextPageToken))
{
nextPageToken = response.NextPageToken;
}
else
{
nextPageToken = null;
}
return messages;
}
public static IList<Message> GetMessage(IList<Message> listMessages)
{
IList<Message> messages = new List<Message>();
foreach (Message listMessage in listMessages)
{
UsersResource.MessagesResource.GetRequest request = GmailService.Users.Messages.Get("me", listMessage.Id);
request.Format = UsersResource.MessagesResource.GetRequest.FormatEnum.Raw;
messages.Add(request.execute());
}
return messages;
}
Related
The session window in Flink is not working as expected on prod env (same logic works on local env). The idea is to emit the count of 'sample_event_two' for a specific user Id & record id incase if there is at least one event of type 'sample_event_one' for the same user Id & record id. ProcessingTimeSessionWindows with session gap of 30 mins is used here and ProcessWindowFunction has the below logic (I am doing a keyby user Id and record Id fields before setting the window size),
public void process(
String s,
Context context,
Iterable<SampleEvent> sampleEvents,
Collector<EnrichedSampleEvent> collector)
throws Exception {
EnrichedSampleEvent event = null;
boolean isSampleEventOnePresent = false;
int count = 0;
for (SampleEvent sampleEvent : sampleEvents) {
if (sampleEvent.getEventName().equals("sample_event_one_name")) {
Logger.info("Received sample_event_one for userId: {}");
isSampleEventOnePresent = true;
} else {
// Calculate the count for sample_event_two
count++;
if (Objects.isNull(event)) {
event = new EnrichedSampleEvent();
event.setUserId(sampleEvent.getUserId());
}
}
}
if (isSampleEventOnePresent && Objects.nonNull(event)) {
Logger.info(
"Created EnrichedSampleEvent for userId: {} with count: {}",
event.getUserId(),
event.getCount());
collector.collect(event);
} else if (Objects.nonNull(event)) {
Logger.info(
"No sampleOneEvent event found sampleTwoEvent with userId: {}, count: {}",
event.getUserId(),
count);
}
}
Though there is sample_event_one present in the collection (confirmed by verifying if the log message "Received sample_event_one" was present) and the count is calculated correctly, I don't see any output event getting created. Instead of EnrichedSampleEvent being emitted, I see log message "No sampleOneEvent event found sampleTwoEvent with userID: "123, count: 5". Can someone help me fix this?
Your ProcessWindowFunction will be called for each key individually. Since the key is a combination of user id and record id, it's not enough to know that "Received sample_event_one" appears in the logs for the same user. Even though it was the same user, it might have had a different record id.
I am trying to add a In Memory Caching to my .NET core project that uses EF. I have two queries and want that two query response to be stored in Cache so I dont have to query everytime
var settingscheck = "SELECT TOP 1 [EndTime],[StartTime],[OrderDay]"+
"FROM[dbo].[Settings]"+
"where SUBSTRING(DATENAME(weekday, getdate() AT TIME ZONE 'UTC' AT TIME ZONE 'Eastern Standard Time'), 0, 4) = OrderDay";
var holidaycheck = "SELECT Count(*) FROM[dbo].[HolidayWeeks] where FORMAT(getdate(), 'yyyy-MM-dd') = [HolidateDate]";
I already implemented the Caching to store one of the Query response like below
public async Task<IActionResult> Index(string sortOrder, string searchString,
int? pageNumber, string currentFilter)
{
int holidaycheck;
var timeUtc = DateTime.UtcNow;
var easternZone = TimeZoneInfo.FindSystemTimeZoneById("Eastern Standard Time");
var todayDt = TimeZoneInfo.ConvertTimeFromUtc(timeUtc, easternZone);
bool isExist = memoryCache.TryGetValue("HolidayWk", out holidaycheck);
if (!isExist)
{
holidaycheck = (from hc in _context.HolidayWeeks
where hc.HolidateDate.Date == todayDt.Date
select hc).Count();
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetSlidingExpiration(TimeSpan.FromHours(2));
memoryCache.Set("HolidayWk", holidaycheck, cacheEntryOptions);
}
if (holidaycheck != 0)
{
return View("/Views/Customers/AppNotAvailable.cshtml");
}
else
{
but now I am trying to add one more query, I was thinking if I can create the JSON object and add the response from both the queries and cache them in memory so it can be used by the application. I cannot do JOIN because these two queries dont have anything in common I am not sure how to store the response from both the queries in to one JSON object. Or suggest me if there is any other option of doing this. Any help is greatly appreciated
I'm trying to check if any document from my collection contains a Value with a specificTime. But Im doing something wrong.
Im searching for a way to check if any Document in my Collection contains the "DateTime.now().year" in the "CreatedAt" Field.
List documents = [];
#override
void initState() {
Firestore.instance.collection('Collection').getDocuments().then((snapshot) {
List<DocumentSnapshot> allDocs = snapshot.documents;
List<DocumentSnapshot> yearDocs = allDocs.where((e) => e.data['createdAt'].toDate.year() == DateTime.now().year).toList();
// iterate over your yearDocs, and add the data to documents
yearDocs.forEach((item){
documents.add(item);
});
});
super.initState();
}
I want to set a List with Documents where the "Field" contains the "DateTime.now().year". Then i check if the List is emtpty.
documents.length >= 1? Container(
child: Text('works'),
) : SizedBox(),
Answer 3.0
Your answer lies in the following
Fetch the createdAt element from the DataSnapshot array which is allDocs
There is a catch that createdAt brings in the timestamp in the Firebase TimeStamp instance format, that is => TimeStamp(seconds=int, nanoseconds=int), for example: Timestamp(seconds=1593504718, nanoseconds=706167000)
What we care about is getting the seconds, which is nothing but a timestamp only
Use the seconds and get the Year from it, and store it in your yearDocs
To get the data in the form of year, we do this
allDocs.forEach((item) => print(item.data["createdAt"].seconds));
// gives your timestamp
// 1593504718
// 1593504699
// 1593259892
// 1596121191
We can get the year via doing this
new DateTime.fromMillisecondsSinceEpoch(1593504718 * 1000).year
Combining all of these, let us code:
List documents = [];
#override
void initState() {
Firestore.instance.collection('Collection').getDocuments().then((snapshot) {
List<DocumentSnapshot> allDocs = snapshot.documents;
// now here we get the data in respect of year
List<DocumentSnapshot> yearDocs = allDocs.where((e) =>
DateTime.fromMillisecondsSinceEpoch(e.data["createdAt"].seconds * 1000).year == DateTime.now().year).toList();
print(yearDocs.length);
// iterate over your yearDocs, and add the data to documents
yearDocs.forEach((item){
documents.add(item);
});
print(documents.length);
});
}
Rest will be fine, you can populate the data as per the documents.length. Please let me know if that helped you. :) This will also help you gain some insight => DateTime class
I think you’re missing data
yearDocs = allDocs.where((e) => e.data['createdAt'].toDate.year() == DateTime.now()).toList();
I have written an APEX Class that sends an email when a client is released. There is a method that I thought I had bulkified but I was told it does not. This is because this method calls another function which actually does the actual email creation and that is not bulkified. Can someone guide me as to how the SOQL queries can be taken out of the method?
global class LM_ChangeAccountRT {
private static final Profile sysAdmin = [select id from profile where name='System Administrator'];
#AuraEnabled
public static String resendEmails(List<String> accountIdList) {
String response = null;
try {
//Only send emails if user is either an ARMS Administor or System Administrator
if (System.label.ARMS_Administrator_Profile_Id == userinfo.getProfileId() ||
sysAdmin.Id == userinfo.getProfileId()) {
List<Account> accList = [SELECT Id,Client_Released__c, RecordTypeId,Client_Number__c, Client_Released__c, Email_Sent__c FROM Account WHERE Id IN:accountIdList];
for(Account acc: accList){
if (acc.Client_Number__c != null && acc.Client_Released__c && acc.Email_Sent__c == true) {
sendpdfgenerationEmails(acc); //this is the method thats not bulkified.
acc.Email_Sent__c = false;
response = 'Email Sent';
}else {
response= 'Access Denied';
}
}
update accList;
}
}catch(Exception e) {
System.debug(e.getMessage());
response = 'Error sending emails';
}
return response;
}
public static void sendpdfgenerationEmails(Account acc){
system.debug('start of confirmation card and pdf generation');
//Logic to find which VF template is used to send an email.
list<EmailTemplate> templateId = new list<EmailTemplate>();
string temppartner;
String partner_opt_in_attachment;
boolean sendFCAmail;
List<Dealership_PDF_Generation__c> custsettingdata = Dealership_PDF_Generation__c.getall().values();
System.debug('custom setting size = ' + custsettingdata.size());
// Fetch State
if(acc.Dealership_State__c!=null && acc.Dealership_Partner__c!=null)
{
for(Dealership_PDF_Generation__c tempcustsetting :custsettingdata)
{
if(acc.Dealership_Partner__c == tempcustsetting.Dealership_Partner__c && acc.Dealership_State__c==tempcustsetting.State__c && tempcustsetting.State__c=='WA' && acc.Dealership_State__c=='WA'){
//For WA State
// temppartner= '%' + tempcustsetting.TEMPLATE_Unique_name__c + '%';
temppartner= tempcustsetting.TEMPLATE_Unique_name__c;
if(acc.Dealership_Spiff_Payment__c == '% premium'){
partner_opt_in_attachment=tempcustsetting.opt_in_form_premium__c;
}else{
partner_opt_in_attachment=tempcustsetting.opt_in_form_nonpremium__c;
}
}
else if(acc.Dealership_Partner__c == tempcustsetting.Dealership_Partner__c && acc.Dealership_State__c==tempcustsetting.State__c && tempcustsetting.State__c=='TX' && acc.Dealership_State__c=='TX'){
//For TX State
//temppartner= '%' + tempcustsetting.TEMPLATE_Unique_name__c + '%';
temppartner= tempcustsetting.TEMPLATE_Unique_name__c;
if(acc.Dealership_Spiff_Payment__c == '% premium'){
partner_opt_in_attachment=tempcustsetting.opt_in_form_premium__c;
}else{
partner_opt_in_attachment=tempcustsetting.opt_in_form_nonpremium__c;
}
}
else if(acc.Dealership_Partner__c == tempcustsetting.Dealership_Partner__c && acc.Dealership_State__c!=tempcustsetting.State__c && tempcustsetting.State__c!='TX' && acc.Dealership_State__c!='TX' && acc.Dealership_State__c!='WA' &&tempcustsetting.State__c!='WA' ){
//For Non TX State
//temppartner= '%' + tempcustsetting.TEMPLATE_Unique_name__c + '%';
temppartner= tempcustsetting.TEMPLATE_Unique_name__c;
if(acc.Dealership_Spiff_Payment__c == '% premium'){
partner_opt_in_attachment=tempcustsetting.opt_in_form_premium__c;
}else{
partner_opt_in_attachment=tempcustsetting.opt_in_form_nonpremium__c;
}
system.debug('grabbed template: ' + temppartner);
}
if(acc.Dealership_Partner__c != null && temppartner!=null ){
templateId.add([Select id,DeveloperName from EmailTemplate where DeveloperName = :temppartner]); //This will probably cause governor limit issues. First problem
}
if (partner_opt_in_attachment != null) {
StaticResource sr = [Select s.Name, s.Id, s.Body From StaticResource s where s.Name =: partner_opt_in_attachment]; //'static_resource' is the name of the static resource PDF. This is another SOQL query that will cause problems
Blob tempBlob = sr.Body;
Messaging.EmailFileAttachment efa = new Messaging.EmailFileAttachment();
efa.setBody(tempBlob);
efa.setFileName('Opt-in.pdf');
List<Messaging.EmailFileAttachment> attachments = new List<Messaging.EmailFileAttachment>();
attachments.add(efa);
// add attachment to each email
for (Messaging.SingleEmailMessage email : emails) {
email.setFileAttachments(attachments);
}
}
system.debug('email sent: ' + emails.size());
Messaging.sendEmail(emails);
}
}
}
The reason why I am trying to bulkify this is because I have written a APEX scheduler that calls the resendemails method everyday at 7am to check which records need to have an email sent. I am afraid that if there are more than a 100 clients then it will cause problems and not send the emails. Any suggestions on how I can optimize the sendpdfemailgeenration() method?
Thank you
Yes, you are right - your's resendEmails() method is not bulkified.
Firstly, let me explain you why is that:
SOQL to get Accounts
Loop 1 on List of Account records
Call sendpdfgenerationEmails() method
Retrieve list of Dealership_PDF_Generation__c records
Loop 2 on List of Dealership_PDF_Generation__c records
SOQL to get StaticResources - Very bad! It's inside double loop!
Call Messaging.sendEmail() method - Very bad! It's inside double loop!
Update on List of Account records
You need to remember that:
1. You should never do SOQLs in loops! - Limit 100 SOQLs per transaction
2. You should never call Messaging.sendEmail() in loops! - Limit 10 calls per transaction
Now let me guide you how to refactor this method:
#AuraEnabled
public static String resendEmails(List<String> accountIdList) {
// 1. SOQL for List of Account records
// 2. Retrieve list of Dealership_PDF_Generation__c records
// 3. SOQL for List of StaticResources for all Names from Dealership_PDF_Generation__c records
// 4. Declaration of new List variable for Messaging.SingleEmailMessage objects
// 5. Loop 1 on List of Account records
// 6. Call new "prepareEmailsForAccount()" method, which prepares and returns list of Messaging.SingleEmailMessage objects
// 7. Add returned Messaging.SingleEmailMessage objects to list from point 4
// 8. End of loop 1
// 9. Call "Messaging.sendEmail()" method with list from point 4
// 10. Update on List of Account records
}
With this you will avoid SOQLs and calling Messaging.sendEmail() method in loops.
I am trying to use a cursor with Objectify and Google App Engine to return a subset of data and a cursor so that I can retrieve more data when the user is ready. I found an example here that looks exactly like what I need but I don't know how to return the final list plus the cursor. Here is the code I have:
#ApiMethod(name = "listIconThemeCursor") //https://code.google.com/p/objectify-appengine/wiki/Queries#Cursors
public CollectionResponse<IconTheme> listIconThemeCursor(#Named("cursor") String cursorStr) {
Query<IconTheme> query = ofy().load().type(IconTheme.class).limit(10);
if (cursorStr != null ) {
query.startAt(Cursor.fromWebSafeString(cursorStr));
}
List<IconTheme> result = new ArrayList<IconTheme>();
int count = 0;
QueryResultIterator<IconTheme> iterator = query.iterator();
while (iterator.hasNext()) {
IconTheme theme = iterator.next();
result.add(theme);
count++;
}
Cursor cursor = iterator.getCursor();
String encodeCursor = cursor.toWebSafeString();
return serial(tClass, result, encodeCursor);
}
Note that this was modified from a previous endpoint in which I returned the CollectionResponse of ALL the data. My dataset is large enough that this is no longer practical. Basically, I don't know what was in the user's function of 'serial(tClass, result, encodeCursor) that let it get returned to the user.
There is another example here but it doesn't appear to answer my question either.
I don't quite understand what you are asking, but I see one immediate bug in your code:
query.startAt(Cursor.fromWebSafeString(cursorStr));
...should be:
query = query.startAt(Cursor.fromWebSafeString(cursorStr));
Objectify command objects are immutable, functional objects.
After a long slog, I figured out that CollectionResponse has the cursor in it :(
Here is the complete code I used incorporating the comment from stickfigure above:
#ApiMethod(name = "listIconThemeCursor", path="get_cursor")
public CollectionResponse<IconTheme> listIconThemeCursor(#Named("cursor") String cursorStr) {
Query<IconTheme> query = ofy().load().type(IconTheme.class)
.filter("errors <", 10)
.limit(10);
if (cursorStr != null ) {
query = query.startAt(Cursor.fromWebSafeString(cursorStr));
}
List<IconTheme> result = new ArrayList<IconTheme>();
QueryResultIterator<IconTheme> iterator = query.iterator();
while (iterator.hasNext()) {
IconTheme theme = iterator.next();
result.add(theme);
}
Cursor cursor = iterator.getCursor();
CollectionResponse<IconTheme> response = CollectionResponse.<IconTheme> builder()
.setItems(result)
.setNextPageToken(cursor.toWebSafeString())
.build();
return response;
}