I would like to compare dates using a sumo logic query. Those dates are sent once per day (So sumologic log receive dates are not useful in this case) and the field is called "date". That field contains the exact date of the triggered event following this format:
date:"YYYY-MM-DDTHH:MM:SSZ"
In order to have visibility about the difference between the current events and the last 24hs events I think the first step is to get yesterday's events; however, I don't know how to do that comparison. I have the following:
| json "field1", "date" as field1, date
| where field1 matches "MATCH_RULE" and date <= 2
| count by field1
So I expect to get access to the yesterday events. As a next step, I will make a difference between that events number and the current ones.
Related
I have a column in Google Data Studio, which looks like this:
Date
Rating
NULL
NULL
NULL
NULL
NULL
NULL
NULL
NULL
2022-01-01 11:44:19
9
2022-01-03 06:03:26
3
2022-02-03 06:03:26
4
2022-02-03 13:39:52
5
2022-03-03 13:41:33
2
The desired date format is dd/mm/yyyy (I don't really need the HMS). I'm trying to get the sum of each rating by month.
Sample data is here.
Sample report is here.
The "NULL" is not actually a null value but text. Because of this, the entire column is being treated as a text field. Is there a function that would ignore the "NULL" text values and only consider the dates, thus treating the field as a date format?
A three step approach is to first create a new date field (titled Date_Calc below), then filter out NULL values and finally change the field type from the default Date & Time to Date at the data source-level and to Month at the chart-level:
1) Date_Calc
Create the data source-level calculated field below which uses the PARSE_DATETIME function to "convert text to a date with time", with the input %F %T, where %F represent "the date in the format %Y-%m-%d" and %T is "the time in the format %H:%M:%S"; additionally, the function ensures that other values such as the text "NULL" would be converted to NULL values:
PARSE_DATETIME("%F %T", Date)
2) Filter
Exclude Date_Calc Is NULL
3) Additional Changes
This section looks at a core requirement (field type) as well as a few optional changes that could be made at the data source-level:
Field Type: Change the field type of the Date_Calc field from Date & Time to Date at the data source-level; the granularity can then further be changed at the chart-level (such as to Quarter, Year, Year Month or in this case, Month)
Hide Fields: The current text field, Date could be hidden at the data source
Rename Fields: The hidden Date field could be renamed to Date_Original and the Date_Calc field could be renamed to Date
Editable Google Data Studio Report (Embedded Google Sheets Data Source) and a GIF to elaborate:
Consider create a calculated field that has the following formula:
IF(LENGTH(Date)=4,"",Date)
Where "4" is the lenght of "NULL" word.
The previous formula checks the length of the Date field and (if the lenght is the same as the lenght of the NULL value), then, set a empty string1; otherwise, keep the original value of the Date value.
In your report example, I've created a field called NewField - which has the described formula above.
1 Instead of an empty string, you can set a default value, like 0001-01-01 or set another default value according to your needs.
Every day I check which events happened. Each event is represented as a document in Solr index.
An event can happen 4 days in a row, skips a day and then happen the next 4 days and so on... In a typical search query, I need to find all events that happened between start date and end date and also return the date ranges of event occurrence.
In the image above, a search between 1. and 17. would return R1(1-4), R2(6-9) and R3(11-14)
How do you store multiple date ranges in indexed and stored solr field?
Using angularjs, dynamodb as DB here.
I have a form where user saves some data. I save my "CreateOn" date in my dynamo db as:
DateTime.UtcNow.ToString("o");
//This saves date in DB as:2018-08-21T12:58:08.7823906Z
Storing like this because dynamo db requires dates (string) to stored in ISO8601 format if you want to use between operator to search for date range.
Now I have a search filters on my page which is basically an angular calendar. When the user selects the date in the calendar( start and end date) I want to get the data back based on the selected date. Here I am using moment to pass the calendar selected date to my api call as:
moment(createdOn).toISOString()
Eg: If they select the Today's date in the calendar I pass the selected date
(Tue Aug 21 2018 00:00:00 GMT-0400 (Eastern Daylight Time)) to the above function
The result of passing this date to moment(createdOn).toISOString() is
2018-08-21T04:00:00.000Z
The search condition at dynamo db is:
conditions.Add(new ScanCondition("CreatedOn", ScanOperator.Between, startDate, endDate ));
If the user selects from the calendar the start date as "08-20-2018" (2018-08-20T04:00:00.000Z) and the end date is "08-21-2018"(2018-08-21T04:00:00.000Z), the code all the data created b/w these 2 dates.
Now the issue is if they select same start and end date then the code does not returns any data, I believe because the start and end date is "2018-08-21T04:00:00.000Z" and the time part of this is all 0000 etc.
My question is how can I convert the date from my calendar ie my end date to correctly reflect the the end time which they select. I havent used ISO8601 format before so not sure how can I do so.
Thanks
You don't need moment for this. You can zero out the time using a Date in the following way.
const date = new Date('2018-08-21T12:58:08.7823906Z')
date.setUTCHours(0)
date.setUTCMinutes(0)
date.setUTCSeconds(0)
date.setUTCMilliseconds(0)
Then you can simply use toISOString() to format the date.
date.toISOString()
// returns '2018-08-21T00:00:00.000Z'
If you don't want to zero out the time, and instead want to set some specific time, you can use a similar approach, just substitute the 0 with whatever time you want.
Some other things to note: DynamoDB doesn't require any specific formatting for dates. DynamoDB simply does a string or number comparison depending on what the field is defined as. You could store your dates in DynamoDB as integers or another string format if you feel that would be easier to work with.
Also, I'm not sure how your table is setup but make sure that your "CreateOn" field is the Range key and that you are using Query, not Scan. Using the Scan operation doesn't scale well.
In Solr, if you have an indexed piece of data, and within that data you had a set of date values, how can you query against the index and ask for events between X and Y date?
For example, if I have a list of Event Venues, each with dozens of events (single, all day, or multi-day), how would you construct the filter to return venues whos events are between the start and end date specified in a search?
Right now, if I search in a form and submit it through to Solr, the query string looks like this:
&fq=dm_event_start_date\:value:["2013-01-04T05:00:00Z" TO *]
&fq=dm_event_end_date\:value:[* TO "2013-01-08T05:00:00Z"]
&fq=bm_tickets_left\:value:"TRUE"
What I really want to ask for are events that occur or start on January 4th, don't last beyond January 8th, AND still have tickets left.
I feel like what I am getting in return is any event that either falls between the two dates, or has tickets available- not necessarily matching the dates.
Probably the date field values needs to be out of the quote in the range query e.g. :-
&fq=dm_event_start_date:value:[2013-01-04T05:00:00Z TO *]
&fq=dm_event_end_date:value:[* TO 2013-01-08T05:00:00Z]
&fq=bm_tickets_left:value:true
Also the bm_tickets_left needs to be just the string value.
I have a daily record table where records are stored date wise. I am using hibernate criteria to access data. How do i get the last date till which records are present continuously (date wise continuity) by providing a date range. For example, say records are there from 21-09-2012 to 25-09-2012 , again from 27-09-2012 to 31-09-2012. I want to form a query using criteria to get record of date 25-09-2012 because for 26-09-2012 there are no records (by passing date ge 21-09-2012 and date le 31-09-2012) . I want to know the last date till which records are present continuously. Say the table has three fields - 1.recordId (AI) 2.date 3.Integer record.
Its not a proper solution to your question. But it may be scenario specific.
How about getting the data for a date range and show then on a calender. Change the color of date if the corresponding value is null.
I think HQL will be better way to this in Hibernate:
http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/queryhql.html