Type Mistmatch Scala Slick Query - database

I am having trouble to get username and password using slick in scala like basically similar to something like
var query = "SELECT * FROM \"default\".users as A " + "WHERE " + " A.username LIKE \'" + email + "\' " + "AND" + " A.password LIKE \'" + password + "\' ";
Here is my case class for schema
case class User(
id: Long = 0L,
username: String,
password: String,
author_id:Long,
created_on: DateTime,
updated_by: Long,
updated_on:Option[DateTime]
)
class UserTable(tag:Tag) extends Table[User](tag,"user"){
override def * = (id,username,password,author_id,created_on,updated_by,updated_on.?) <> (User.tupled,User.unapply)
def id = column[Long]("id",O.PrimaryKey,O.AutoInc)
def username = column[String]("username")
def password = column[String]("password")
def created_on = column[DateTime]("created_on")
def updated_on = column[DateTime]("dateupdated")
def author_id = column[Long]("author_id")
def updated_by = column[Long]("updated_by")
}
lazy val UserTable = TableQuery[UserTable]
Below is my Query
val users = Main.UserTable.filter(_.username == email)
.filter(_.password == password).take(1)

filter(_.username == email)
You probably meant === instead of ==.
That is part of the Slick DSL to build query expressions (and they could not call it == because Scala's equality comparator cannot be replaced).
Warning: Most operators mimic the plain Scala equivalents, but you have to use === instead of == for comparing two values for equality and =!= instead of != for inequality. This is necessary because these operators are already defined (with unsuitable types and semantics) on the base type Any, so they cannot be replaced by extension methods.

Related

NIFI - upload binary.zip to SQL Server as varbinary

I am trying to upload a binary.zip to SQL Server as varbinary type column content.
Target Table:
CREATE TABLE myTable ( zipFile varbinary(MAX) );
My NIFI Flow is very simple:
-> GetFile:
filter:binary.zip
-> UpdateAttribute:<br>
sql.args.1.type = -3 # as varbinary according to JDBC types enumeration
sql.args.1.value = ??? # I don't know what to put here ! (I've triying everything!)
sql.args.1.format= ??? # Is It required? I triyed 'hex'
-> PutSQL:<br>
SQLstatement= INSERT INTO myTable (zip_file) VALUES (?);
What should I put in sql.args.1.value?
I think it should be the flowfile payload, but it would work as part of the INSERT in the PutSQL? Not by the moment!
Thanks!
SOLUTION UPDATE:
Based on https://issues.apache.org/jira/browse/NIFI-8052
(Consider I'm sending some data as attribute parameter)
import java.nio.charset.StandardCharsets
import org.apache.nifi.controller.ControllerService
import groovy.sql.Sql
def flowFile = session.get()
def lookup = context.controllerServiceLookup
def dbServiceName = flowFile.getAttribute('DatabaseConnectionPoolName')
def tableName = flowFile.getAttribute('table_name')
def fieldName = flowFile.getAttribute('field_name')
def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find
{ cs -> lookup.getControllerServiceName(cs) == dbServiceName }
def conn = lookup.getControllerService(dbcpServiceId)?.getConnection()
def sql = new Sql(conn)
flowFile.read{ rawIn->
def parms = [rawIn ]
sql.executeInsert "INSERT INTO " + tableName + " (date, "+ fieldName + ") VALUES (CAST( GETDATE() AS Date ) , ?) ", parms
}
conn?.close()
if(!flowFile) return
session.transfer(flowFile, REL_SUCCESS)
session.commit()
maybe there is a nifi native way to insert blob however you could use ExecuteGroovyScript instead of UpdateAttribute and PutSQL
add SQL.mydb parameter on the level of processor and link it to required DBCP pool.
use following script body:
def ff=session.get()
if(!ff)return
def statement = "INSERT INTO myTable (zip_file) VALUES (:p_zip_file)"
def params = [
p_zip_file: SQL.mydb.BLOB(ff.read()) //cast flow file content as BLOB sql type
]
SQL.mydb.executeInsert(params, statement) //committed automatically on flow file success
//transfer to success without changes
REL_SUCCESS << ff
inside the script SQL.mydb is a reference to groovy.sql.Sql oblject

Packages to use in python for the creation of a database table comparison tool

I was tasked with developing a tool that will accept a few parameters and then query 2 databases based on a list of tables.
There are 3 possible database options, a connection to Netezza, a connection to Oracle, or a connection to a DB2 Mainframe. In theory they will pass me the type of connection, hostname, port, database name, username, and password.
The query will take a table from the list, query both databases and compare the data in the table across the 2 DBs.
For the connection to Netezza i am using pyodbc, for the connection to Oracle i am using cx_oracle, and for the connection to DB2 i am using ibm_db.
At the moment i was able to make a connection to each and i was able to return the column metadata of the table in each db as well as a result set from each.
There are a few things i am trying to accomplish.
If the column is of a certain data type (i.e. decimal, integer) i want to sum all the values for that column in the table, if it is of any other datatype (i.e. string, date) i want to count do a count().
I would like to do this for the table in both DBs and then do a comparison of the column counts/totals and display the comparison in excel.
Finally i would like to do a column by column comparison of every row in the table in both DBs. If there are any differences in the field values for each row then the entire row will be displayed in an excel spreadsheet.
What i am wondering is if there are any packages in python that i can use to perform these table like operations.
Please see the code below for what i have so far.
import pyodbc
import ibm_db
import cx_Oracle
import collections
class DatabaseConnection(object):
def __init__(self, connection_type, hostname_or_ip, port, database_or_sid, username, password):
self.port = port
self.connection_type = connection_type
self.hostname_or_ip = hostname_or_ip
self.database_or_sid = database_or_sid
self.username = username
self.password = password
self.dsn = "GEMPROD"
self.connection_string = ""
self.conn = ""
def __enter__(self):
if self.connection_type == "Netezza":
self.connection_string = "DRIVER={NetezzaSQL};SERVER=" + self.hostname_or_ip + ";PORT="+ self.port + \
";DATABASE=" + self.database_or_sid + ";UID=" + self.username + ";PWD=" + self.password
self.conn = pyodbc.connect(self.connection_string)
return self.conn
elif self.connection_type == "Oracle":
dsn_tns = cx_Oracle.makedsn(self.hostname_or_ip, self.port, self.database_or_sid)
self.conn = cx_Oracle.connect(user=self.username, password=self.password, dsn=dsn_tns)
return self.conn
elif self.connection_type == "DB2":
self.connection_string = "Database=" + self.database_or_sid + ";HOSTNAME=" + self.hostname_or_ip + \
";PORT=" + self.port + ";PROTOCOL=TCPIP;UID=" + self.username + ";PWD=" + \
self.password + ";"
#self.conn = ibm_db.connect(self.connection_string, "", "")
self.conn = ibm_db.connect('DSN=' + self.dsn, self.username, self.password)
return self.conn
pass
def __exit__(self, type, value, traceback):
if self.connection_type == "Netezza":
self.conn.close()
elif self.connection_type == "DB2":
ibm_db.close(self.conn)
elif self.connection_type == "Oracle":
self.conn.close
pass
def __repr__(self):
return '%s%s' % (self.__class__.__name__, self.dsn)
def query(self, query, params):
pass
#database_column_metadata = collections.namedtuple('DatabaseColumnMetadata','index column_name data_type')
#database_field = collections.namedtuple('', '')
table_list = ['BNR_CIF_25DAY_RPT', table2]
sort_column = None
with DatabaseConnection('Netezza', ip, port, database, username, pwd) as connection_one:
print('Netezza Query:')
for table in table_list:
cursor = connection_one.cursor()
netezza_rows = cursor.execute("SELECT * FROM BNR_CIF_25DAY_RPT LIMIT 1")
column_list = netezza_rows.description
sort_column = str(column_list[0][0])
netezza_query = "SELECT * FROM BNR_CIF_25DAY_RPT ORDER BY " + sort_column + " ASC LIMIT 10"
netezza_rows = cursor.execute(netezza_query)
print(column_list)
netezza_column_list = []
for idx, column in enumerate(column_list):
column_name, data_type, *rest = column
netezza_column_list.append((idx, column_name, data_type))
for row in netezza_rows:
print(row, end='\n')
for tup in netezza_column_list:
print(tup, end='\n')
print('Netezza row count:', str(netezza_rows.rowcount) + '\n')
cursor.close()
with DatabaseConnection('Oracle', hostname, port, SID, username, pwd) as connection_two:
print('Oracle Query:')
for table in table_list:
try:
cursor = connection_two.cursor()
oracle_rows = cursor.execute("SELECT * FROM BNR_CIF_25DAY_RPT WHERE ROWNUM <= 1")
column_list = oracle_rows.description
sort_column = column_list[0][0]
oracle_query = "SELECT * FROM (SELECT * FROM BNR_CIF_25DAY_RPT ORDER BY " + sort_column + " ASC) WHERE ROWNUM <=10"
oracle_rows = cursor.execute(oracle_query)
print(column_list)
oracle_column_list = []
for idx, column in enumerate(column_list):
column_name, data_type, *rest = column
oracle_column_list.append((idx, column_name, data_type))
for row in oracle_rows:
print(row, end='\n')
for tup in oracle_column_list:
print(tup, end='\n')
print('Oracle row count:', str(oracle_rows.rowcount) + '\n')
except cx_Oracle.DatabaseError as e:
print(str(e))
finally:
cursor.close()
Apologize for anything that didnt make sense and the poor code as i am new to Python and program is still in it's infancy.
This is not exactly python based solution but we used to do it in our shop to compare netezza and Oracle using fluid query . Fluid query

How to use IN Clause for list of strings or GUID's in Dapper

I am trying to write a dapper query for IN clause, but it's not working throwing casting error saying "Conversion failed when converting the nvarchar value 'A8B08B50-2930-42DC-9DAA-776AC7810A0A' to data type int." . In below query fleetAsset is Guid converted into string.
public IQueryable<MarketTransaction> GetMarketTransactions(int fleetId, int userId, int rowCount)
{
//Original EF queries which I am trying to convert to Dapper
//var fleetAsset = (from logicalFleetNode in _context.LogicalFleetNodes
// where logicalFleetNode.LogicalFleetId == fleetId
// select logicalFleetNode.AssetID).ToList();
////This query fetches guid of assetprofiles for which user having permissions based on the assets user looking onto fleet
//var assetProfileIds = (from ap in _context.AssetProfileJoinAccounts
// where fleetAsset.Contains(ap.AssetProfile.AssetID) && ap.AccountId == userId
// select ap.AssetProfileId).ToList();
var fleetAsset = _context.Database.Connection.Query<string>("SELECT CONVERT(varchar(36),AssetID) from LogicalFleetNodes Where LogicalFleetId=#Fleetid",
new { fleetId }).AsEnumerable();
//This query fetches guid of assetprofiles for which user having permissions based on the assets user looking onto fleet
var sql = String.Format("SELECT TOP(#RowCount) AssetProfileId FROM [AssetProfileJoinAccounts] AS APJA WHERE ( EXISTS (SELECT " +
"1 AS [C1] FROM [dbo].[LogicalFleetNodes] AS LFN " +
"INNER JOIN [dbo].[AssetProfile] AS AP ON [LFN].[AssetID] = [AP].[AssetID]" +
" WHERE ([APJA].[AssetProfileId] = [AP].[ID]) " +
" AND ([APJA].[AccountId] = #AccountId AND LogicalFleetId IN #FleetId)))");
var assetProfileIds = _context.Database.Connection.Query<Guid>(sql, new { AccountId = userId, FleetId = fleetAsset, RowCount=rowCount });
Dapper performs expansion, so if the data types match, you should just need to do:
LogicalFleetId IN #FleetId
(note no parentheses)
Passing in a FleetId (typically via an anonymous type like in the question) that is an obvious array or list or similar.
If it isn't working when you remove the parentheses, then there are two questions to ask:
what is the column type of LocalFleetId?
what is the declared type of the local variable fleetAsset (that you are passing in as FleetId)?
Update: test case showing it working fine:
public void GuidIn_SO_24177902()
{
// invent and populate
Guid a = Guid.NewGuid(), b = Guid.NewGuid(),
c = Guid.NewGuid(), d = Guid.NewGuid();
connection.Execute("create table #foo (i int, g uniqueidentifier)");
connection.Execute("insert #foo(i,g) values(#i,#g)",
new[] { new { i = 1, g = a }, new { i = 2, g = b },
new { i = 3, g = c },new { i = 4, g = d }});
// check that rows 2&3 yield guids b&c
var guids = connection.Query<Guid>("select g from #foo where i in (2,3)")
.ToArray();
guids.Length.Equals(2);
guids.Contains(a).Equals(false);
guids.Contains(b).Equals(true);
guids.Contains(c).Equals(true);
guids.Contains(d).Equals(false);
// in query on the guids
var rows = connection.Query(
"select * from #foo where g in #guids order by i", new { guids })
.Select(row => new { i = (int)row.i, g = (Guid)row.g }).ToArray();
rows.Length.Equals(2);
rows[0].i.Equals(2);
rows[0].g.Equals(b);
rows[1].i.Equals(3);
rows[1].g.Equals(c);
}

Searching in database from a form returning empty input

I want to do search in my database using some fields filled by a form
but if some fields are left empty i don't want to include them
which kind of query can help me in achieving this??
Currently i am using a query like:
Select * from DATABASE where COLUMN='form_input';
but as my form will return empty it will try and select rows which have null entries but rather i want to this time see a result of all rows in database i.e i want to invalidate the filter by COLUMN='form_input'
As we do not know your server side scripting language -
The psheuducode should be -
if(request['form_input']!=null)
Select * from DATABASE where COLUMN='form_input';
else
Select * from DATABASE;
Also If there are many fields for form_input then we can design
our code something like -
String wherequery = "";
if(request['form_input1']!=null)
{
wherequery = wherequery + " COLUMN='form_input1' ";
}
if(request['form_input2']!=null)
{
wherequery = wherequery + " And "
wherequery = wherequery + " COLUMN='form_input2' ";
}
if(request['form_input3']!=null)
{
wherequery = wherequery + " And "
wherequery = wherequery + " COLUMN='form_input3' ";
}
....
And so on
....
String selectQuery = "";
if(wherequery == "")
{
selectQuery = "Select * from TABLE";
}
else
{
selectQuery = "Select * from TABLE where" + wherequery ;
}
execute (selectQuery);
Please note we are using pseudo code here. We can take the form inputs and concatenate query for each input which is not null.
If we find the concatenated string as blank string, we will select the full table.
Otherwise
we will select with the where clause query.
Hope, this help you out.

How to count number of rows in SQL Server database?

I am trying to count the number of unread messages in my DB Table but is proving to be very difficult. I've even read tutorials online but to no avail.
What I'm doing should be simple.
Here's what I'm trying to do:
COUNT NUMBER OF ROWS IN NOTIFICATIONSTABLE
WHERE USERID = #0 AND MESSAGEWASREAD = FALSE
Can somebody please point me in the right direction? Any help will be appreciated.
Thank you
#helper RetrievePhotoWithName(int userid)
{
var database = Database.Open("SC");
var name = database.QuerySingle("select FirstName, LastName, ProfilePicture from UserProfile where UserId = #0", userid);
var notifications = database.Query("SELECT COUNT(*) as 'counter' FROM Notifications WHERE UserID = #0 AND [Read] = #1", userid, false);
var DisplayName = "";
if(notifications["counter"] < 1)
{
DisplayName = name["FirstName"] + " " + name["LastName"];
}
else
{
DisplayName = name["FirstName"] + ", you have " + notifications["counter"] + " new messages.";
}
<img src="#Href("~/Shared/Assets/Images/" + name["ProfilePicture"] + ".png")" id="MiniProfilePicture" /> #DisplayName
database.Close();
}
SELECT COUNT(*) FROM NotificationsTable WHERE
UserID = #UserID AND MessageWasRead = 0;
Sql Count Function
Okay so this is based on what I think should be done. I don't know the underlying types, so it is going to be my best guess.
var notifications = database.QuerySingle("Select COUNT(*) as NumRecs....");
if((int)notifications["NumRecs"] > 0)) .......
I changed the query for notifications to QuerySingle. You don't need a recordest, you only need a scalar value, so that should (hopefully remove your problem with the implicit conversion in the equals you were having.
I would also check to see if your database object implements IDisposable (place it in a using statement if so) as you are calling close, and this won't actually call close (I know it's not dispose but it might have dispose too) if you encounter and exception before the close function is called.
int unreadMessageCount = db.Query("SELECT * FROM Notification WHERE UserId=#0 AND Read=#1",UserId,false).Count();
string displayname = name["FirstName"] + " " + name["LastName"] + unreadMessageCount>0?",you have " + unreadMessageCount :"";

Resources