I'm using Scala 2.10 with Slick 1.0.0 and trying to do a lifted query.
I have a table, "Logins", where I'm attempting to do a load, and groupBy on a Timestamp column. However, when I attempt to groupBy, I am running into an issue when I try and format the Timestamp field to extract only the day portion, to group the objects by the same day.
Given the objects:
id | requestTimestamp
1 | Jan 1, 2013 01:02:003
2 | Jan 1, 2013 03:04:005
3 | Jan 1, 2013 05:06:007
4 | Jan 2, 2013 01:01:001
I'd like to return a grouping out of the database by similar days, where, for the sake of brevity, the the following Formatted timestamp to id relationship happens, where the id's would actually be a list of objects
Jan 1, 2013 -> (1, 2, 3)
Jan 2, 2013 (4)
I've got the following slick table object:
private implicit object Logins extends Table[(Int, Timestamp)]("LOGINS") {
def id = column[Int]("ID", O.PrimaryKey)
def requestTimeStamp = column[Timestamp]("REQUESTTIMESTAMP", O.NotNull)
def * = logId ~ requestTimeStamp
}
The following Query method:
val q = for {
l <- Logins if (l.id >= 1 && l.id <= 4)
} yield l
val dayGroupBy = new java.text.SimpleDateFormat("MM/dd/yyyy")
val q1 = q.groupBy(l => dayGroupBy.format(l.requestTimeStamp))
db.withSession {
q1.list
}
However, instead of getting the expected grouping, I get an exception on the line where I attempt the groupBy:
java.lang.IllegalArgumentException: Cannot format given Object as a Date
Does anyone have any suggestions on properly grouping by Timestamps out of the database?
Timestamp and Date are not the same thing! Try to convert Timestamp to Human understandable text using calendar or SimpleDateTime.
Not so sure about the second one though!
Related
Imagine I had a 'horizontal' data set that contained:
Unique Key
Multiple 'pairs' of dates across multiple columns (i.e. Event A Start, Event B Start, Event C Start, etc and separate columns for Event A End, Event B End, Event C End, etc).
A single date (not a pair) for a specific 'Event'.
In essence, looks something like this:
Data Set
Unique Key
Event A Start
Event A End
Single Date Event
Event B Start
Event B End
2nd Single Date Event
Key 1
1 Jan 2021
3 Jan 2021
2 Jan 2021
5 Jan 2021
10 Jan 2021
10 Jan 2021
Key 2
7 Jan 2021
10 Jan 2021
null
null
null
null
How would I convert the Data Set above into a table like this using PowerQuery?
Expected Output:
Unique Key
Event
Start Date
End Date
Key 1
Event A
1 Jan 2021
3 Jan 2021
Key 1
Single Date Event
null
2 Jan 2021
Key 1
Event B
5 Jan 2021
10 Jan 2021
Key 1
2nd Single Date Event
null
10 Jan 2021
Key 2
Event A
7 Jan 2021
10 Jan 2021
I've tried:
Unpivot but I can't rename both "Event A Start" and "Event A End" into "Event A". I even tried renaming all "Event [x] Start" as "Event [x]", did a 'unpivot selected' of all "Event [x]'. Then I renamed all "Event [x] End" into "Event [x]" and then performed an unpivot on those columns. Unfortunately, the Key and Event columns don't line up.
Merge Query: I have tried merging one query with another but it's not quite getting the desired output. I created two separate queries (one with Key, Event, and Start Date; another with Key, Event and End Date). But this not having the desired effect. I think this is because of the Single Date Events being 'null'?
I feel I am definitely doing something wrong, so asking here to see if the output that I want is even achievable with PowerQuery based on the input data?
You definitely have to do a bit of extra work on top of an unpivot.
Here's how I'd approach it:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8k6tVDBU0lEyVPBKzFMwMjACcYyROUbIHFNkjqEBLl6sDsRkI6C4OW4teaU5Odip2FgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Unique Key" = _t, #"Event A Start" = _t, #"Event A End" = _t, #"Single Date Event" = _t, #"Event B Start" = _t, #"Event B End" = _t, #"2nd Single Date Event" = _t]),
#"Unpivoted Columns" = Table.UnpivotOtherColumns(Source, {"Unique Key"}, "Event", "Value"),
#"Changed Type" = Table.TransformColumnTypes(#"Unpivoted Columns",{{"Value", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each if Text.Contains([Event], "Start") then "Start Date" else "End Date"),
#"Transformed Text" = Table.TransformColumns(#"Added Custom",{{"Event", each if Text.EndsWith(_, "Start") or Text.EndsWith(_, "End") then Text.BeforeDelimiter(_, " ", {0, RelativePosition.FromEnd}) else _, type text}}),
#"Pivoted Column" = Table.Pivot(#"Transformed Text", List.Distinct(#"Transformed Text"[Custom]), "Custom", "Value")
in
#"Pivoted Column"
Steps:
Unpivot the date columns
Add a new column to tag each row as Start Date / End Date
Strip off " Start" / " End" suffix in the [Event] column
Pivot on the new column from Step 2
I'm new to SQL, please Help me how to convert this query into LinQ
This is My Table Dept:
Id Name Sal Department
1 John 40000 Dotnet
2 mick 45000 DotNet
3 Pillay 777 Sql
Here I want to display Salary Based On Department Name, like:
DepartmentName ToalSal
Dotnet 85000
Sql 777
select DeprtmentName,sum(sal) from Dept_Emp Group by DeprtmentName
I wrote some Part of query
public IEnumerable<Dept_Emp> GetJam()
{
var x = from n in db.Dept_Emp
group n by n.Sal into g
select new
{
DeprtmentName = g.Key
};
// what I mention Here;
}
You are missing calculating sum of sal fields of grouped entities. Also you are grouping by wrong field. You should use department name for grouping
from de in db.Dept_Emp
group de by de.DeprtmentName into g
select new {
DepartmentName = g.Key,
TotalSalary = g.Sum(x => x.Sal) // aggregation here
}
What you have as output is anonymous objects. You cannot return them directly from method. You have several options here
Create custom class like DepartmentTotals with name and total salary fields, and return instances of this class instead of anonymous objects. Then return type will be IEnumerable<DepartmentTotals>.
Create Tuple<string, int> (or whatever type of salary). And return such tuples.
Use C# 7 tuples.
I have some objects in db. Every object has parameter Date in date format like a "Mon Oct 05 08:55:36 CEST 2015". I want load all objects from 24h before to present time. I am using hibernate entity manager. Can you help me please
Here is my entity params:
private Long id;
private String email;
private String answer1;
private String answer2;
private Date date;
Here is table schema:
ID ; email ; answer1 ; answer2 ; date
1 ; any#thing.com ; 1234 ; 4321 ; 2015-10-04 18:01:19
I assume then, that the type of the date table column is indeed SQL DATE. In that case, you can simply run a HQL/JPQL query
SELECT e FROM YourEntity e WHERE e.date BETWEEN :from AND :to
Make a PreparedStatement out of it and provide the desired values for the from and to parameters. Sample using JPA:
EntityManager em = ...;
Date to = new Date();
Calendar fromCalendar = Calendar.getInstance();
fromCalendar.setTime(to);
fromCalendar.add(Calendar.HOUR_OF_DAY, -24);
Date from = fromCalendar.getTime();
em
.createQuery("SELECT e FROM YourEntity e WHERE e.date BETWEEN :from AND :to", YourEntity.class)
.setParameter("from", from, TemporalType.TIMESTAMP)
.setParameter("to", to, TemporalType.TIMESTAMP)
.getResultList();
Using JodaTime or Java8 Time package may make the whole date manipulation easier (didn't know if you were using those, so I gave an example using the "basic" Java classes).
My products have date_created and expires_in attributes. date_created is Magento date backend format while expires_in is a select that contains options as 1 week, 1 month etc.
Using those two attributes I'm trying to determine the total number of expired products. My idea was to write a query that selects:
products where date created + 1 week < now() AND expires_in = 1 week
AND
products where date created + 1 month < now() AND expires_in = 1 month
...
But I don't seem to be able to make even the first part of the first step to work:
$currentTimestamp = Mage::getModel('core/date')->timestamp(time());
$oneWeekTimestamp = Mage::getModel('core/date')->timestamp(strtotime("+1 week"));
$products = Mage::getResourceModel('catalog/product_collection')
->setStoreId(Mage::app()->getStore()->getId())
->addAttributeToSelect('')
...
//these are what i have tried so far:
$products->addAttributeToFilter('date_updated + ' . $oneWeekTimestamp , array('gt' => $currentTimestamp));
$products->addAttributeToFilter(new Zend_Db_Expr('date_updated + ' . $oneWeekTimestamp), array('gt' => $currentTimestamp));
$products->addAttributeToFilter("DATEDIFF($currentTimestamp, 'date_updated')" , array('gt' => 7));
$countExpired = $products -> count();
None of them seem to work. If possible I'd like the cross RDBMS solution.
You can do this in following way:
$currentTimestamp = Mage::getModel('core/date')->timestamp(time());
$oneWeekTimestamp = Mage::getModel('core/date')->timestamp(strtotime("+1 week"));
$oneweekDate=date("Y-m-d H:i:s",$oneWeekTimestamp);
$products->addAttributeToFilter('date_updated',array('date'=>true,'gt',$oneweekDate);
I have a table in Google Datastore that holds n values in n columns, and one of them is a timestamp.
The timestamp property is defined like this, inside the table class (Java):
#Persistent
private Date timestamp;
The table is like this:
id | value | timestamp
----------------------------------------------------------
1 | ABC | 2014-02-02 21:07:40.822000
2 | CDE | 2014-02-02 22:07:40.000000
3 | EFG |
4 | GHI | 2014-02-02 21:07:40.822000
5 | IJK |
6 | KLM | 2014-01-02 21:07:40.822000
The timestamp column was added later to the table, so some rows have not the corresponding timestamp value.
I'm trying, using Python Google App Engine to build an api that returns the total number of rows that have a timestamp >= to some value.
For example:
-- This is just an example
SELECT * FROM myTable WHERE timestamp >= '2014-02-02 21:07:40.822000'
I've made this class, in python:
import sys
...
import webapp2
from google.appengine.ext import db
class myTable(db.Model):
value = db.StringProperty()
timestamp = datetime.datetime
class countHandler(webapp2.RequestHandler):
def get(self, tablename, timestamp):
table = db.GqlQuery("SELECT __key__ FROM " + tablename + " WHERE timestamp >= :1", timestamp )
recordsCount = 0
for p in table:
recordsCount += 1
self.response.out.write("Records count for table " + tablename + ": " + str(recordsCount))
app = webapp2.WSGIApplication([
('/count/(.*)/(.*)', countHandler)
], debug=True)
I've successfully deployed it and I'm able to call it, but for some reason I don't understand it's always saying
Records count for table myTable: 0
I'm struggling with the data type for the timestamp.. I think the issue is there.. any idea? which type should it be declared?
Thank you!
You problem (as discussed in the comments as well) seems to be that you are passing a string (probably) to the GqlQuery parameters.
In order to filter your query by datetime you need to pass a datetime object in to the query params. For that take a look here on how to convert that.
Small example:
# not sure how your timestamps are formatted but supposing they are strings
# of eg 2014-02-02 21:07:40.822000
timestamp = datetime.datetime.strptime(timestamp, "%Y-%m-%d %H:%M:%S.%f" )
table = db.GqlQuery("SELECT __key__ FROM " + tablename + " WHERE timestamp >= :1", timestamp)