https://cloud.google.com/appengine/docs/standard/python/config/cron
A cron handler is just a normal handler defined in app.yaml. You can
prevent users from accessing URLs used by scheduled tasks by
restricting access to administrator accounts.
However this option is not available in nodejs. What's the recommended way to protect a CRON URL from public access with the nodejs runtime?
Check if your request contains
X-Appengine-Cron: true
The X-Appengine-Cron header is set internally by Google App Engine. If
your request handler finds this header it can trust that the request
is a cron request. If the header is present in an external user
request to your app, it is stripped, except for requests from logged
in administrators of the application, who are allowed to set the
header for testing purposes.
https://cloud.google.com/appengine/docs/flexible/nodejs/scheduling-jobs-with-cron-yaml#validating_cron_requests
This can be done e.g. like this:
app.get('/my-cron-trigger', async (req, res, next) => {
// ensure that request comes from cron
if (req.header('x-appengine-cron') !== 'true') {
next();
return;
}
// [your function code]
}
Related
I'm attempting to secure a .Net 6.0 / Razor Page web application against Azure AD. I was able to complete the application registration with Azure AD and successfully authenticate users. The issue I'm facing occurs when the issued token expires. I have some experience working with Angular and IdentityServer implementations, but Razor Page/Microsoft Identity is still new to me.
What I would like to happen:
The user logs in with their Microsoft account
The user's session is uninterrupted for up to 12 hours (with all token management happening behind the scenes)
After 12 hours the session/cookies will expire and the user will need to log in again
What is happening:
The user logs in and is authenticated
After approximately one hour, the application triggers a call to the /authorize endpoint the next time the user takes any action (such as trying to navigate to a new page)
This causes the application to reload on the page the user was currently on (thus interrupting their experience)
Additional Issue: I am also receiving a CORS error under similar circumstances as above. The difference here is this is occurring when the user is in the middle of form data entry when the (presumed) token expiration occurs. When they click submit to post the form, a 302 xhr / Redirect to the /authorize endpoint is triggered. This call results in a CORS error. Refreshing the page is required to trigger a successful call (and they need to start over on their form). Update: This is occurring due to an AJAX call (nothing to do with the form/post specifically). See the edit at the end.
Ideally, I would like the token to be automatically (and silently) refreshed via a refresh token once it is nearing expiration. I would also, of course, like to avoid the scenario of the CORS error when they are attempting to post when the token has expired.
Some code snippets (note: I'm manually adding authentication to an existing app, I did not use any scaffolding/templates for the initial project creation).
Note: I initially tried the below implementation without defining custom authOptions, but during debugging and different attempts at resolution, it exists in the below state. Results were consistent either way.
Program.cs
var builder = WebApplication.CreateBuilder(args);
var config = builder.Configuration;
var services = builder.Services;
services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
.AddMicrosoftIdentityWebApp(
authOptions =>
{
config.Bind("AzureAD", authOptions);
authOptions.MaxAge = TimeSpan.FromHours(12);
authOptions.SaveTokens = true;
},
sessionOptions =>
{
sessionOptions.Cookie.MaxAge = TimeSpan.FromHours(12);
sessionOptions.Cookie.Name = "Custom-Cookie-Name";
sessionOptions.ExpireTimeSpan = TimeSpan.FromHours(12);
sessionOptions.SlidingExpiration = false;
})
.EnableTokenAcquisitionToCallDownstreamApi(config.GetValue<string>("GraphApi:Scopes")?.Split(' '))
.AddMicrosoftGraph(config.GetSection("GraphApi"))
.AddSessionTokenCaches();
services.AddRazorPages(options =>
{
options.Conventions.AddPageRoute("/Disclaimer", "/");
})
.AddMvcOptions(options =>
{
var policy = new AuthorizationPolicyBuilder().RequireAuthenticatedUser().Build();
options.Filters.Add(new AuthorizeFilter(policy));
});
services.AddHttpContextAccessor();
........
var app = builder.Build();
if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Error");
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseSession();
app.UseAuthentication();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
endpoints.MapRazorPages();
});
app.UseSaveUserDetailsOnAuthentication();
app.UseIdentityPageInitialization();
app.MapRazorPages();
app.MapControllers();
app.Run();
I also have some middleware that is using the graph service to hit the /me endpoint and store some user details under specific conditions (in case this is relevant):
Graph Middleware
public async Task InvokeAsync(HttpContext context, UserManager<ApplicationUser> userManager, GraphServiceClient graphServiceClient)
{
var page = context.GetRouteValue("page")?.ToString();
if (!page.IsNullOrEmpty() && page.Equals("/Disclaimer") && context.User.Identity?.IsAuthenticated == true)
{
var user = await graphServiceClient.Me
.Request()
.GetAsync()
.ConfigureAwait(false);
The below snippet is what occurs when attempting the post scenario above.
The tl/dr questions are, using the Microsoft Identity libray/MSAL, how do I:
Silently refresh a user's token
Avoid reloading the page to get a new token (i.e.: calling /authorize and redirecting to obtain a new token)
Handle token expiration from the client-side (avoid the CORS error when posting a form). Do I need to add an additionally client-side js library to manage this?
I've tried scouring Microsoft's documentation, but nothing I've found goes into detail on this. The closest I found was MSAL's documentation mentioning that it handles token refresh for you (but it seemingly isn't happening in my case).
I'm expecting that the token will be silently refreshed by the underlying MSAL library, but that does not appear to be happening. Additionally, I'm expecting to avoid CORS errors on the front-end related to token expiration.
EDIT: While my main question still remains, I believe I found the resolution for the secondary issue: the CORS issue which is actually triggered via an AJAX call to the API. This article outlines that Microsoft.Identity.Web v1.2.0+ now handles this scenario. I now have a vague idea on how to handle it, but still need to attempt the implementation.
I found a reference here explaining that these session token caches have a scoped lifetime and should not be used when TokenAcquisition is used as a singleton, which I believe is the case with the use of the Microsoft Graph API ("AddMicrosoftGraph").
I switched the session token cache to a distributed SQL token cache. However, I do not believe any of this was actually the root issue.
I've identified an issue causing my server (clustered behind a LB without sticky sessions) encryption keys to not be correctly stored/shared in a distributed store. What was happening is any idle timeout in ISS would reset them, causing the auth cookie to be unusable. Additionally, any time the app would hit a different web server behind the LB, the existing auth cookie to be unusable by the new server (because they were using separate keys). So in both scenarios the application would redirect the user for authentication.
The fix for this was simply implementing a distributed key store as described here. The provided stores did not work for me, due to restrictions put in place by my client, so I just implemented a custom IXmlRepository and registered it:
services.Configure<KeyManagementOptions>(options => options.XmlRepository = new CustomXmlRepository());
So at the end of the day I had the following issues:
The auth cookie was becoming invalidated due to changing/lost keys as described above: Resolved by adding a distributed key store
The Microsoft GraphServiceClient was unable to obtain access tokens/refresh tokens (resulting in MSAL errors), due to a lack of a distributed token store as well as due to changing/lost keys (when I was storing tokens in the cookies): Resolved by adding a distributed token store (described here)
I'm trying to get azure AD authentication working between a Blazor WASM app, and another API that I have running locally but on a different port. I need both applications to use the Azure login, but I only want the user to have to log in once on the Blazor app which should then pass those credentials through to the API.
I've set up app registrations for both apps in the portal, created the redirect url, exposed the API with a scope and I can successfully log into the blazor app and see my name using #context.User.Identity.Name.
When it then tries to call the API though, I get a 401 error back and it doesn't hit any breakpoints in the API (presumably because there is no authentication being passed across in the http request).
My code in the Blazor app sets up a http client with the base address set to the API:
public static async Task Main(string[] args)
{
var builder = WebAssemblyHostBuilder.CreateDefault(args);
builder.RootComponents.Add<App>("#app");
builder.Services.AddHttpClient("APIClient", client => client.BaseAddress = new Uri("https://localhost:11001"))
.AddHttpMessageHandler<BaseAddressAuthorizationMessageHandler>();
builder.Services.AddScoped(sp => sp.GetRequiredService<IHttpClientFactory>().CreateClient("APIClient"));
builder.Services.AddMsalAuthentication<RemoteAuthenticationState, CustomUserAccount>(options =>
{
builder.Configuration.Bind("AzureAd", options.ProviderOptions.Authentication);
options.ProviderOptions.DefaultAccessTokenScopes.Add("api://d3152e51-9f5e-4ff7-85f2-8df5df5e2b2e/MyAPI");
//options.UserOptions.RoleClaim = "appRole";
});
await builder.Build().RunAsync();
}
In my API, I just have the Authorise attribute set on the class, and eventually will need roles in there too:
[Authorize]
[Route("api/[controller]")]
[ApiController]
public class CarController
Then, in my Blazor component, I then inject the http factory and try to make a request:
#inject IHttpClientFactory _factory
...
private async Task RetrieveCars()
{
var httpClient = _factory.CreateClient("APIClient");
HttpResponseMessage response = await httpClient.GetAsync("https://localhost:11001/api/cars");
var resp = await response.Content.ReadAsStringAsync();
cars = JsonSerializer.Deserialize<List<Car>>(resp);
}
but this returns the 401 error. I've also tried a few different variations like just injecting a http client (#inject HttpClient Http) but nothing seems to be adding my authorisation into the API calls. The options.UserOptions.RoleClaim is also commented out in the AddMsalAuthentication section as I wasn't sure if it was needed, but it doesn't work with or without it in there.
Can anyone explain what I'm doing wrong and what code I should be using?
Common causes.
Most cases ,we tend to forget to grant consent after giving API
permissions in the app registration portal,after exposing the api
which may lead to unauthorized error.
Other thing is when Audience doesn’t match the “aud” claim when we
track the token in jwt.io .Make sure ,Audience=clientId is configured
in the code in authentication scheme or Token validation parameters
by giving ValidAudiences.And also try with and without api:// prefix
in client id parameter.
Sometimes aud claim doesn’t match as we mistakenly send ID token
instead of Access tokens as access tokens are meant to call APIs .So
make sure you check mark both ID Token and access token in portal
while app registration.
While Enabling the authentication by injecting the [Authorize]
attribute to the Razor pages.Also add reference
Microsoft.AspNetCore.Authorization as(#using
Microsoft.AspNetCore.Authorization)
Please see the note in MS docs and some common-errors
If above are not the cases, please provide with additional error details and startup configurations or any link that you are following to investigate further.
I'm using Google App Engine to host a couple of services (a NextJS SSR service and a backend API built on Express). I've setup my dispatch.yaml file to route /api/* requests to my API service and all other requests get routed to the default (NextJS) service.
dispatch:
- url: '*/api/*'
service: api
The problem: I've also turned on Identity-Aware Proxy for App Engine. When I try to make a GET request from my NextJS service to my API (server-side, via getServerSideProps) it triggers the IAP sign-in page again instead of hitting my API. I've tried out a few ideas to resolve this:
Forwarding all cookies in the API request
Setting the X-Requested-With header as mentioned here
Giving IAP-secured Web App User permissions to my App Engine default service account
But nothing seems to work. I've confirmed that turning off IAP for App Engine allows everything to function as expected. Any requests to the API from the frontend also work as expected. Is there a solution I'm missing or a workaround for this?
You need to perform a service to service call. That's no so simple and you have not really example for that. Anyway I tested (in Go) and it worked.
Firstly, based your development on the Cloud Run Service to Service documentation page.
You will have this piece of code in NodeJS sorry, I'm not a NodeJS developer and far least a NexJS developer, you will have to adapt
// Make sure to `npm install --save request-promise` or add the dependency to your package.json
const request = require('request-promise');
const receivingServiceURL = ...
// Set up metadata server request
// See https://cloud.google.com/compute/docs/instances/verifying-instance-identity#request_signature
const metadataServerTokenURL = 'http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience=';
const tokenRequestOptions = {
uri: metadataServerTokenURL + receivingServiceURL,
headers: {
'Metadata-Flavor': 'Google'
}
};
// Fetch the token, then provide the token in the request to the receiving service
request(tokenRequestOptions)
.then((token) => {
return request(receivingServiceURL).auth(null, null, true, token)
})
.then((response) => {
res.status(200).send(response);
})
.catch((error) => {
res.status(400).send(error);
});
This example won't work because you need the correct audience. Here, the variable is receivingServiceURL. It's correct for Cloud Run (and Cloud Functions) but not for App Engine behind IAP. You need to use the Client ID of the OAuth2 credential named IAP-App-Engine-app
Ok, hard to understand what I'm talking about. So, go to the console, API & Services -> Creentials. From there, you have a OAuth2 Client ID section. copy the Client ID column of the line IAP-App-Engine-app, like that
Final point, be sure that your App Engine default service account has the authorization to access to IAP. And add it as IAP-secured Web App User. The service account has this format <PROJECT_ID>#appspot.gserviceaccount.com
Not really clear also. So, go to the IAP page (Security -> Identity Aware Proxy), click on the check box in front of App Engine and go the right side of the page, in the permission panel
In the same time, I can explain how to deactivate IAP on a specific service (as proposed by NoCommandLine). Just a remark: deactivate security when you have trouble with it is never a good idea!!
Technically, you can't deactive IAP on a service. But you can grant allUsers as IAP-secured Web App User on a specific service (instead of clicking on the checkbox of App Engine, click on the checkbox of a specific service). And like that, even with IAP you authorized all users to access to your service. it's an activation without checks in fact.
I want to associate static files with particular users in express and place some kind of authentication so that a particular set of files can only be seen by the user they are associated with.
I am using multer to upload files.
My middleware serves all files requested.
app.use(express.static(path.join(__dirname, "public")));
In my SQL database I'm storing the path of the image along with user_id. Now when a resource is requested I want to make sure whether the requesting user owns it otherwise reject the request.
You can add a middleware function right before it and have that middleware check to see if the request is permitted - something like this.
app.use(checkUser, express.static(path.join(__dirname, "public")));
checkUser would then need to be a middleware function that checks to see if the user is allowed to request this particular URL. If not, then it returns an error and does NOT call next() so express.static() never sees the request. If the user is allowed, then the middleware calls next() and express.static() will see if it can serve that URL.
I am trying to invoke a Cloud Run service using Cloud Tasks as described in the docs here.
I have a running Cloud Run service. If I make the service publicly accessible, it behaves as expected.
I have created a cloud queue and I schedule the cloud task with a local script. This one is using my own account. The script looks like this
from google.cloud import tasks_v2
client = tasks_v2.CloudTasksClient()
project = 'my-project'
queue = 'my-queue'
location = 'europe-west1'
url = 'https://url_to_my_service'
parent = client.queue_path(project, location, queue)
task = {
'http_request': {
'http_method': 'GET',
'url': url,
'oidc_token': {
'service_account_email': 'my-service-account#my-project.iam.gserviceaccount.com'
}
}
}
response = client.create_task(parent, task)
print('Created task {}'.format(response.name))
I see the task appear in the queue, but it fails and retries immediately. The reason for this (by checking the logs) is that the Cloud Run service returns a 401 response.
My own user has the roles "Service Account Token Creator" and "Service Account User". It doesn't have the "Cloud Tasks Enqueuer" explicitly, but since I am able to create the task in the queue, I guess I have inherited the required permissions.
The service account "my-service-account#my-project.iam.gserviceaccount.com" (which I use in the task to get the OIDC token) has - amongst others - the following roles:
Cloud Tasks Enqueuer (Although I don't think it needs this one as I'm creating the task with my own account)
Cloud Tasks Task Runner
Cloud Tasks Viewer
Service Account Token Creator (I'm not sure whether this should be added to my own account - the one who schedules the task - or to the service account that should perform the call to Cloud Run)
Service Account User (same here)
Cloud Run Invoker
So I did a dirty trick: I created a key file for the service account, downloaded it locally and impersonated locally by adding an account to my gcloud config with the key file. Next, I run
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://url_to_my_service
That works! (By the way, it also works when I switch back to my own account)
Final tests: if I remove the oidc_token from the task when creating the task, I get a 403 response from Cloud Run! Not a 401...
If I remove the "Cloud Run Invoker" role from the service account and try again locally with curl, I also get a 403 instead of a 401.
If I finally make the Cloud Run service publicly accessible, everything works.
So, it seems that the Cloud Task fails to generate a token for the service account to authenticate properly at the Cloud Run service.
What am I missing?
I had the same issue here was my fix:
Diagnosis: Generating OIDC tokens currently does not support custom domains in the audience parameter. I was using a custom domain for my cloud run service (https://my-service.my-domain.com) instead of the cloud run generated url (found in the cloud run service dashboard) that looks like this: https://XXXXXX.run.app
Masking behavior: In the task being enqueued to Cloud Tasks, If the audience field for the oidc_token is not explicitly set then the target url from the task is used to set the audience in the request for the OIDC token.
In my case this meant that enqueueing a task to be sent to the target https://my-service.my-domain.com/resource the audience for the generating the OIDC token was set to my custom domain https://my-service.my-domain.com/resource. Since custom domains are not supported when generating OIDC tokens, I was receiving 401 not authorized responses from the target service.
My fix: Explicitly populate the audience with the Cloud Run generated URL, so that a valid token is issued. In my client I was able to globally set the audience for all tasks targeting a given service with the base url: 'audience' : 'https://XXXXXX.run.app'. This generated a valid token. I did not need to change the url of the target resource itself. The resource stayed the same: 'url' : 'https://my-service.my-domain.com/resource'
More Reading:
I've run into this problem before when setting up service-to-service authentication: Google Cloud Run Authentication Service-to-Service
1.I created a private cloud run service using this code:
import os
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route('/index', methods=['GET', 'POST'])
def hello_world():
target = os.environ.get('TARGET', 'World')
print(target)
return str(request.data)
if __name__ == "__main__":
app.run(debug=True,host='0.0.0.0',port=int(os.environ.get('PORT', 8080)))
2.I created a service account with --role=roles/run.invoker that I will associate with the cloud task
gcloud iam service-accounts create SERVICE-ACCOUNT_NAME \
--display-name "DISPLAYED-SERVICE-ACCOUNT_NAME"
gcloud iam service-accounts list
gcloud run services add-iam-policy-binding SERVICE \
--member=serviceAccount:SERVICE-ACCOUNT_NAME#PROJECT-ID.iam.gserviceaccount.com \
--role=roles/run.invoker
3.I created a queue
gcloud tasks queues create my-queue
4.I create a test.py
from google.cloud import tasks_v2
from google.protobuf import timestamp_pb2
import datetime
# Create a client.
client = tasks_v2.CloudTasksClient()
# TODO(developer): Uncomment these lines and replace with your values.
project = 'your-project'
queue = 'your-queue'
location = 'europe-west2' # app engine locations
url = 'https://helloworld/index'
payload = 'Hello from the Cloud Task'
# Construct the fully qualified queue name.
parent = client.queue_path(project, location, queue)
# Construct the request body.
task = {
'http_request': { # Specify the type of request.
'http_method': 'POST',
'url': url, # The full url path that the task will be sent to.
'oidc_token': {
'service_account_email': "your-service-account"
},
'headers' : {
'Content-Type': 'application/json',
}
}
}
# Convert "seconds from now" into an rfc3339 datetime string.
d = datetime.datetime.utcnow() + datetime.timedelta(seconds=60)
# Create Timestamp protobuf.
timestamp = timestamp_pb2.Timestamp()
timestamp.FromDatetime(d)
# Add the timestamp to the tasks.
task['schedule_time'] = timestamp
task['name'] = 'projects/your-project/locations/app-engine-loacation/queues/your-queue/tasks/your-task'
converted_payload = payload.encode()
# Add the payload to the request.
task['http_request']['body'] = converted_payload
# Use the client to build and send the task.
response = client.create_task(parent, task)
print('Created task {}'.format(response.name))
#return response
5.I run the code in Google Cloud Shell with my user account which has Owner role.
6.The response received has the form:
Created task projects/your-project/locations/app-engine-loacation/queues/your-queue/tasks/your-task
7.Check the logs, success
The next day I am no longer able to reproduce this issue. I can reproduce the 403 responses by removing the Cloud Run Invoker role, but I no longer get 401 responses with exactly the same code as yesterday.
I guess this was a temporary issue on Google's side?
Also, I noticed that it takes some time before updated policies are actually in place (1 to 2 minutes).
For those like me, struggling through documentation and stackoverflow when having continuous UNAUTHORIZED responses on Cloud Tasks HTTP requests:
As was written in thread, you better provide audience for oidcToken you send to CloudTasks. Ensure your requested url exactly equals to your resource.
For instance, if you have Cloud Function named my-awesome-cloud-function and your task request url is https://REGION-PROJECT-ID.cloudfunctions.net/my-awesome-cloud-function/api/v1/hello, you need to ensure, that you set function url itself.
{
serviceAccountEmail: SERVICE-ACCOUNT_NAME#PROJECT-ID.iam.gserviceaccount.com,
audience: https://REGION-PROJECT-ID.cloudfunctions.net/my-awesome-cloud-function
}
Otherwise seems full url is used and leads to an error.