I'm using Prisma Client to manage the database. and I found the problem while i'm making front-end.
In Prisma Client, there is a method "connect" and it is really powerful technic to map between tables.
However, for example, I connect three records like A, B, C, by order C, A, B, but when request data to check how it looks, it actually shows like A, B, C.
Is there any way not to sort when using connect?
Sorting by default would happen on id of A, B, C which is by default and you can configure it using orderBy in include.
Related
I have a particularly challenging situation that I could use some assistance with.
I work for a manufacturing facility and am working on a proof of concept.
I have a number of client devices (PIs) fixed to manufacturing equipment, all collecting data from the equipment and storing this data locally within an embedded MySQL database on the device. I would like to import the data from each of the devices, into a central Microsoft SQL Data Warehouse. I would prefer this to be pulled from the devices by the server, rather than being pushed from the client devices.
I would then like the embedded database on the device to be updated / purged, to prevent the same data from being resent (initially I was thinking a date field in a table which I just timestamp once that record has been copied).
My feelings are that a SSIS package would be the way to go here, I have IP addresses and connection information for the PIs in a table within the DW, and so would like to connect to each client in turn to import the data and update it.
Is there a way to change a connection string on the fly within SSIS? OR would there be a better way to achieve this - maybe via a sproc on the DW?
I'm ok with sprocs, but very new to SSIS. If you have any links/tutorials/posts that may help, please share. Thanks.
EDIT: This is what I already have
Here are my variables:
As you can see it is showing an error when attempting to run on the first step.
Also, FWIW, here's the progress output...
Is there a way to change a connection string on the fly within SSIS?
Use a variable to store the connection string, and use that variable to populate the Expression value of the connection string. Then when you change the value of the variable, you will change the value of the connection string.
Its not the answer but something like this.
A) you create a table all the IP address, and connection strings.
B) SSIS create variables for each property i.e Variable IPAddress.
C) Create Execute SQL Task; Set Full Result set.
Also Add Result Set: Result Name: 0 Varaible Name Rows.
D) Create another Variable Rows: DataType System.Object.
E) Add Foreach Loop Container: ADO: Rows
Variable Mapping: IPAddress
F) Create Source Connection Manager
Expression set the connection as of your Variables
G) Add a Data Flow Task and fetch the data from each connection.
Say I have around 10-20GB of data in HDFS as a Hive table. This has been obtained after several Map-Reduce jobs and JOIN over two separate datasets. I need to make this Queryable to the user. What options do I have?
Use Sqoop to transfer data from HDFS to an RDS like Postgresql. But I want to avoid spending so much time on data transfer. I just tested HDFS->RDS in the same AWS region using Sqoop, and 800mb of data takes 4-8 minutes. So you can imagine ~60gb of data would be pretty unmanagable. This would be my last resort.
Query Hive directly from my Webserver as per user request. I haven't ever head of Hive being used like this so I'm skeptical about this. This struck me because I just found out you can query hive tables remotely after some port forwarding on the EMR cluster. But being new to big(ish) data I'm not quite sure about the risks associated with this. Is it commonplace to do this?
Some other solution - How do people usually do this kind of thing? Seems like a pretty common task.
Just for completeness sake, my data looks like this:
id time cat1 cat2 cat3 metrics[200]
A123 1234212133 12 ABC 24 4,55,231,34,556,123....(~200)
.
.
.
(time is epoch)
And my Queries look like this:
select cat1, corr(metrics[2],metrics[3]),corr(metrics[2],metrics[4]),corr(metrics[2],metrics[5]),corr(metrics[2],metrics[6]) from tablename group by cat1;
I need the correlation function, which is why I've chosen postgresql over MySQL.
You have correlation function in Hive:
corr(col1, col2)
Returns the Pearson coefficient of correlation of a pair of a numeric columns in the group.
You can simply connect to a hiveserver port via odbc and execute queries.
Here is an example:
http://www.cloudera.com/content/cloudera/en/downloads/connectors/hive/odbc/hive-odbc-v2-5-10.html
Hive User Experience (hue) has a Beeswax query editor designed specifically for the purpose of exposing Hive to end users who are comfortable with SQL. This way they can potentially run ad-hoc queries against the data residing in Hive without needing to move it elsewhere. You can see an example of the Beeswax Query Editor here: http://demo.gethue.com/beeswax/#query
Will that work for you?
What i can understand from the question posted above is you have some data (20GB ) which you have stored in hdfs and using hive. Now you want to access that data to perform some kind of statistics functions like correlation and others.
You have functions in hive that perform correlation.
Otherwise you can directly connect R to hive using RHive or even excel to hive using datasource.
The other solution is installing hue which comes with hive editors where you can directly query the hive.
In my program, I'm getting username as startup parameter and getting the user-id from server.
Sever and client are connected!
But I need to put the username and the user-id(that belongs to the username) together, send them back to the server and to the end sort them with respect to user-ids. Any suggestions?
Just send them both to the server, taking care to include some kind of delimiter so it's possible to parse the stream and identify which part is which.
For instance, you could separate the two with a NIL-byte, with a final NIL after the userid, too. This assumes that both are strings.
On the server, just store the incoming data into an array of structs, and sort the array with qsort() when you have all the data collected.
I'm new to designing database stuff hands on, and rather than use new features like Linq to Entities code first, I'd like to implement the lower level stuff myself.
If I had a class Foo, which can contain any other number of Foos, it may look something like this:
class Foo
{
int FooId;
List<Foo> containedFoos;
...
}
Suppose the Foo table looks like
FooId, FooName, ContainingFooId (Foreign key to same table)
If I used code first and retrieved all my Foos, their containedFoos list would be automatically populated for me.
My question is, is this done on the applicaiton side (just do a straight select from the Foo table, and code the logic in C# to consruct a Foo, find all its containedFoo, build a list, etc)
or, is there way SQL Server can return the data in a more useful way? Such as returning each Foo and its contained items together in some way?
No, SQL Server won't return the data like that. You would have to create your objects and populate them by hand to mimic the way Code First populates your objects.
This reason alone makes EF very attractive (to me) to use from the beginning of a project. It saves TONS of time.
I've written a C/CGI web application for booking items. My client has requested that I add an 'admin' feature where an authorised user can delete and change data, and those who aren't, can only add data. It is much simpler in concept than most login implementations as there is only a password, and effectively only two states, 'anonymous' and 'admin'.
I come from a PHP background where session management is as simple as session_start(), and I can instantly play around with $_SESSION. Of course, in C/CGI there is nothing built-in like that. I would like to avoid adding a CGI library dependency (I already depend on glib, confuse and libmysqlclient, plus I'm curious to learn about session management).
What is the simplest way to do a password-based session management in C/CGI, without the need for multiple users, large amounts of session data, or anything complex?
A session implies server side maintained state. As you don't have users I guess you want it simpler. If that is the case a signed cookie with an expiration date can do it. This tutorial will show how to do it with Python:
http://webpython.codepoint.net/cgi_cookie
First you have to decide how you are going to persist state in the browser : are you going to use a session cookie or pass a session token on each page ? If you go with the cookie way, you don't have to change your pages and forms, but you need to add cookie management if it's not already present (be careful to use session + httpOnly cookies)
Then you must decide how to get data about the state on the server : if you're already using a database, you could add a "SESSION" table with columns "SESSION_ID" and "EXPIRATION_DATE" + a second table called "SESSION_DATA" with columns "SESSION_ID", "KEY", "VALUE".
You now "just" have to create some simple functions :
int session_createNewSession(long& session_id, long duration)
int session_setValue(long session, char[] key, char[] value)
int session_getValue(long session, char[] key, char[] value)
int session_abandonSession(long session)
This functions would return error codes if session could not be created, or value could not be set/get. You should also probably create a job that runs regularly on the database to delete older sessions.
Now that you have your session system in place, the rest is pretty straightforward :
create a login form
in your cgi handle the received data by checking if the login/password is right (don't store the passwords in the db though : store a salted hash)
if connexion is OK, save the user id in session (or in your case, you could just save a "IsAdmin" value)
You could do in fact simpler : just a session_createNewSession(long& session_id, int isAdmin) would be sufficient in your case (with only one database table), but your client is probably going to ask more features over time isn't he ?
One final note : be careful that your session_id's are random, not indent fields, otherwise it would be quite simple to hijack someone else's session.