How to parse text fields as date in Socrata? - text-parsing

I have some experience with consuming opendata via Socrata platform. The data publisher has unintentionally published his dataset with dates as text datatype in format DD/MM/YYYY
I have managed to select rows with logic like datefield like '%252017' in where clause, but I would like to create a GROUPBY query to group all datefields as years to get a list like
Year - Amount
2016 - 2312
2015 - 12312
2014 - 11372 etc.
Example query: https://opendata.rdw.nl/resource/m9d7-ebf2.xml?$select=count(*)&$limit=50000&$where=voertuigsoort=%27Personenauto%27%20AND%20export_indicator=%27Nee%27%20AND%20merk=%27VOLKSWAGEN%27%20AND%20datum_eerste_toelating%20like%20%27%252017%27

Related

Google Data Studio Table: Dividing Data that has 2 different Years

I need to produce a table that has Quotes win%. The formula is #won divide by #sent.
My problem is, there are quotes that are won within a year but were sent in different years.
(My data comes from BigQuery)
The data looks like this:
Sale Sent Won
sale1 2019 2020
sale2 2019 2020
sale3 2016 2017
sale4 2017 2019
sale5 2020 2020
sale6 2020 2020
sale7 2018 2018
sale8 2016 2016
sale9 2015 2016
sale10 2016 2017
sale11 2016 2018
sale12 2018 2019
I'd like to be able to create a table in data studio like this:
Year SENT WON WIN%
2016 4 2 50%
2017 1 2 200%
2018 2 2 100%
2019 2 2 100%
2020 2 4 200%
I would love to see if this is possible in google data studio. Any suggestion is highly appreciated.
Added a Google Data Studio Report to demonstrate, as well as a GIF showing the process below.
One approach is to restructure the Data at the Data Set and use Calculated Fields in a Table:
1) Data Transformation
The data needs to be transformed from the current Wide structure to a Long data structure. One way it can be achieved in Google Sheets is by using the formula below (Sheet1 represents the input sheet; consult embedded Google Sheet for clarification):
=ArrayFormula(QUERY({
{Sheet1!A:A,IF(LEN(Sheet1!A:A),"Sent",""),Sheet1!B:B};
{Sheet1!A:A,IF(LEN(Sheet1!A:A),"Won",""),Sheet1!C:C}
},"Select * Where Col3 is not null Label Col2 'Dimension', Col3 'Year'",1))
2) Table
- Dimension: Year
- Sort: Year in Ascending order
- Metrics: Add the 3 calculated fields below:
3) Calculated Fields
The formulas below create the metrics used in the Table above (Formula 3.1 and 3.2 need to be added at the Data Source-level, while 3.3 can be added at the Chart-level if required):
3.1) SENT
COUNT(CASE
WHEN REGEXP_MATCH(Dimension, "Sent") THEN Year
ELSE NULL END)
3.2) WON
COUNT(CASE
WHEN REGEXP_MATCH(Dimension, "Won") THEN Year
ELSE NULL END)
3.3) WIN%
WON / SENT

between query for dates not working as i want

i am working on SQL query which take records between two dates "from" To "to" as show in below picture.my query working well but when i change "to" date from 24 to 23 then it does not display the record of 23 date,means last row not display.
my SQL query is given below:
select * from prescription_master where (pr.date between #from_date and #to_date) or (pr.date=#to_date)
i want that the record of 23 date also display when i select 23 date from "to" date picker.for this purpose i use "or (pr.date=#to_date)" in above query but it not working.how i can solve this.
You have to handle time part:
select *
from prescription_master
where (CAST(pr.date AS DATE) between #from_date and #to_date)
or (CAST(pr.date AS DATE) = #to_date)
-- this also will made a query non-SARGable
When you provide parameter the date is set to 23-06-2019 00:00:00 and you are comparing it with 23-06-2019 13:08:00

Entity Converting all DateTimes to Jan 1

I'm using Entity to pull some data from a customer's SQL table and I want to filter it based on date. The customer stored the relevant month of each row as a 6-digit string in the YYYYMM format. My attempt to resolve this (because I can't change the column type to a datetime2) has been to create a SQL View that does the following to create a datetime2 column representing the month:
CONVERT(datetime2, MON.Month + '01') AS CoveredMonth
Then inside of .NET, I have two DateTime objects, yearStart and yearEnd, that represent January 1st, 2016 and January 1st, 2017, as well as the specific employee whose records I'm looking for. I have the following code to attempt to filter on this column:
IList<MonthlyRecord> monthlyRecords = m_LTContext.MonthlyRecords
.Where(r => r.EmployeeID == employee.ID && r.CoveredMonth >= yearStart && r.CoveredMonth < yearEnd)
.ToList<MonthlyRecord>();
When I place a break point to check what is returned to monthlyRecords I see the expected count of records. However, each record has a CoveredMonth set to January 1st, 2016. Running the same query in SQL I get an identical count of records again, but the appropriate dates: Jan 1 '16, Feb 1 '16, etc.
Is there an issue with Entity somehow mapping properties that are non-standard on a View? There a few hacks I have in my mind but I'd really like to resolve this "properly".
Figured it out - the View was using the EmployeeID as the primary key and not the MonthlyRecordID.

Display a yearly summary by month in SQL

I have a table that has a date/time field and am trying to figure out how to run a report so that I can view the sum of the amount between each month separately in the 2012 year.
The table has 2 fields, Amount and TimeStamp, and I'm trying to return a report like this:
January, 2012: $221.20
February, 2012: $150.20
etc etc.
Anyone have any ideas how to accomplish this easily in SQL Server? I want to avoid writing a seperate query for each individual month.
This can be solved by using the MONTH and YEAR Functions on SQL.
SELECT
SUM(Amount) as [Amount]
,MONTH(TimeStamp) as [Month]
,YEAR(TimeStamp) as [Year]
FROM
[MyTable]
GROUP BY
MONTH(TimeStamp)
,YEAR(TimeStamp)

SQL date related

Hi all i want to take record from table named Tblbatch where batch starting date should be from augest 2007 to july 2010...
I want to fetch such records which came in between this two dates
Select * from Tblbatch where startDate between '01-08-2007' and '31-07-2010'
provided you have a datetime column "startDate"
Note : that using between includes both the dates specified.
If you want to avoid the dates either change the boundary dates to + - 1 respectively or use > and < conditions

Resources