What is the recommended way to provide Bigquery access to end users?
We have a multitenant application for appengine where each company has its own namespace in Datastore. We can ingest data from these individual namespaces into BigQuery and run various reports.
We would like to let our users to be able to run queries in their respective BigQuery projects. What is the best way to achieve this?
Possible options:
user sends query to appengine, which passes it to his BigQuery project and returns reply
user sends query to BigQuery directly. The question here, how is the best to faciliate this?
some other way?
My favorite is "user sends query to BigQuery directly. The question here, how is the best to faciliate this?":
BigQuery supports sharing datasets with specific individuals. So you could load each table into its own dataset, and then share that dataset with the individual customers (or even with groups).
You can do this manually on the web UI or, via the REST API:
access[].domain A domain to grant access to. Any users signed in with the domain specified will be granted the specified access. Example: "example.com".
access[].groupByEmail A fully-qualified email address of a mailing list to grant access to. This must be either a Google Groups mailing list (ends in #googlegroups.com) or a group managed by an enterprise version of Google Groups.
access[].userByEmail A fully qualified email address of a user to grant access to. For example: fred#example.com.
access[].role READER, WRITER, OWNER
https://developers.google.com/bigquery/docs/reference/v2/datasets/update
Related
This is my first question, so I hope I don't miss a thing. To be clear from the start: I don't expect an answer which dives deep into detail. This is just about getting a general understanding of how to work with this kind of software.
So I don't know if "Identity Management System" is a suitable term for what I mean but when I talk about Identity Management Systems I think of something like Azure AD, which as far as I know provides e.g. web developers the possibility to integrate a way users can authenticate (including access privilege etc.) on their website.
What I'm quite unsure about is how to work with/ integrate such tools in a project. I will try to make it clear with an example: Assuming I have a website let's say this website is a blog. The blog consist of different posts which are stored in my own database which is connected to the website. The posts are written by different users which authenticate with a tool like Azure AD. The user's data is stored somewhere on a server run by e.g. Microsoft. If I want to display the posts togethere with the name, email.... of the user who wrote them, how would I do this?
Is it possible to query the user's data directly from the Identity Management System and display it? This does not sound ideal to me as the consequence would be that data the website uses is stored in two different locations.
Would you kind of copy the user's data from the Identity Management System to the websites database and query it from there? This does not sound like a good solution either because then data would be duplicated.
So whats the "right workflow"?
I appreciate any hints and further information I can get:-)
AFAIK To get the user's information like name, email etc. you can add these claims while generating the JWT token.
To generate access token, you have multiple authentication flows such as Authorization code flow, ROPC flow, Implicit flow.
To add the claims that you need to return with the token, you can make settings like below:
Go to Azure Portal -> Azure Active Directory -> App Registrations -> Your app -> Token configuration -> Add optional claims
When you decode the token via JSON Web Tokens - jwt.io you can find the user information that you need.
To know how to generate access token, you can refer SO Thread which I solved it before.
I use these following GCP products together for a CRM system:
Cloud SQL
App Engine
Bigquery
Once a week an external application exports data from Bigquery in this way:
The external application makes a request to Appengine with a token.
AppEngine retrieves permissions for this token from Cloud SQL, makes some additional computation to obtain a list of allowed IDs.
Appengine runs a Bigquery's query filtered with these ids. Something like that: SELECT * FROM table WHERE id IN(ids)
Appengine responds to the external application with a unmodified result of query in JSON.
The problem is that the export is not very often, but amount of data can be large and I dont want to load AppEngine with this data. What other GCP products are useful in this case? Remember I need to retrieve permissions from Appengine and CloudSQL.
Unclear whether the JSON is just directly from BigQuery query results, or you do additional processing in the application to render/format it. I'm assuming direct results.
An option that comes to mind is to leverage cloud storage. You can use the signed url feature to provide a time-limited link to your (potential large) results without exposing public access.
This, coupled with BigQuery's ability to export results to GCS (either via an export job, or using the newer EXPORT DATA SQL statement allows you to run a query and deliver results directly to GCS.
With this, you could simply redirect the user to the signed URL at the end of your current flow. There's additional features that are complementary here, such as using GCS data lifecycle features to age out and remove files automatically so you don't need to concern yourself with slow accumulation of results.
We have an application where we store users login name in the format domain\username. We authenticate via windows and then get additional info from our database by matching the domain\username we get from the user to our database.
Now they want to move to the cloud. We authenticate users via apps in Azure AD. However, the user identifier we get back is first.last#domain.com.
I have fiddled around with https://graph.microsoft.com/v1.0/users/email and the select command to try and get the 'old' name. Howev,er I have not yet found out how to get it.
The reason they move to the cloud is that they are merging two ADs. So some users will be DomainA and some DomainB, but in the same tenant. So my first thought was to try and convert the mail to the other format. However, the two different ADs have different naming standards. One has DOMAINA\fila (two first letters from the first name and two first letters from the last name) and the other one has DOMAINB\firlas. Also it feels really ugly to try and solve it that way.
Is it possible to fetch the users loginname formatted as domain\username via Microsoft Graph?
Using the beta edition of Graph, you can obtain the user's domain and username from the onPremisesDomainName and onPremisesSamAccountName properties:
/beta/users?$select=userPrincipalName,onPremisesDomainName,onPremisesSamAccountName
The domain is stored as a FQDN so you'll need to do some translation. For example, domainName.ad.contoso.com might translate to domainName\).
This will give you a workaround so you can match up users with your internal databases. It is however only a temporary solution. Long-term, you really want to migrate to using the userPrincipalName. This is the primary user identifier and guaranteed to be unique within a given tenant.
Azure AD is a little different than the legacy Active Directory. Certain concepts from legacy AD such as Organizational Units (OUs), Group Policy Objects (GPOs), Kerberos Authentication, Lightweight Directory Access Protocol (LDAP), Domain trusts between multiple domains, and several others simply do not exist in the cloud.
I'm testing out the recently released DocumentDb and can't find any documentation indicating best practice on how to perform user data segregation.
I imagine the rough design would be:
Authenticate the user and create new/obtain existing user id
On document insert inject the user id into the document
On read of document/collection of documents query where document user id = current user id
I'm creating an AngularJs application and currently use an Azure Sql Database combined with Azure Mobile Services.
Mobile services handles the user authentication and also the server side user data segregation by the use of data script javascript functions:
e.g.
function insert(item, user, request) {
item.userId = user.userId;
request.execute();
}
Any suggestions on what would be the technique for secure user data segregation from AngularJS using DocumentDB?
Your approach sounds reasonable to me - assuming the logic mentioned in your rough design takes place in your backend service.
Generally, I'd treat DocumentDB similarly as you would treat any other datastore. Your client (AngularJS) makes calls to your backend service, rather than making calls directly to your datastore. Your backend validates the client's request (i.e. assert that the user is authenticated and may touch a particular piece of data) before delegating any work to your datastore.
If direct database access from the client is desired - you can check out DocumentDB's users and permissions. For implementing multi-tenancy for your application, you can create users in DocumentDB which corresponds to your actual users or the tenants of your application. You can then create permissions for a given user which correspond to the access control over various collections, documents, attachments etc. On your client, you can connect to the database using the User's resource key rather than your DocumetnDB's administrator keys.
Check out this blog post on DocumentDB users / permissions: http://blogs.msdn.com/b/cloud_solution_architect/archive/2014/12/09/permissions-in-azure-documentdb.aspx
I have some clients that I'd like to put into Microsoft CRM (3.0 Dynamics). These people are already in a small Active Directory group for access to a couple of internal applications.
Is there a way to add these people to CRM and pull/push the contact data from Active Directory, so I'm not creating a second repository of information that conflict?
Unfortunately there's no out-of-the-box way to dot his. You'd have to write a custom app in order to query AD and pull in the data. Unless you're looking at over 100 customers you probably won't make up the time it would take you to manually input this data.