The restrictions of a https callbackUrl and the nature of the subscriptions as a whole makes it seem like this is something that can only be done with a publicly accessible url.
So far I have come across two potential solutions to make local development / debugging easier.
The first is the Subscription Proxy service offered by google. This workaround essentially lets you remove the SSL restriction and proxy subscription callbacks to a custom URL.
The second and most helpful way I have found to do development locally is to capture a subscription callback request (say from a server that is publicly accessible) into a log and then use curl to reproduce that request on your local/dev machine using something like:
curl -H "Content-type: application/json" -X POST \
-d '{"json for":"the notification"}' http://localhost:8080/notify
Since the requests can sometimes be large, or you might want to test multiple callback types, I also found it useful to put the JSON of the subscript request into various files (ex: timeline-respond.json) and then run
curl -H "Content-Type: application/json" \
--data #timeline-respond.json http://localhost:8080/notify
I'm curious as to what other people are doing to test their application subscriptions locally.
The command line curl technique that you mention is the best I've found to date.
I've experimented with other solutions, such as an App Engine subscription target paired with a local script which pulls that App Engine service for new notifications to relay to localhost, but so far I haven't found one that's worth the added complexity.
Alternatively, there are many localhost proxies available. My favorite is ngrok.com.
You might want to give localtunnel a try.
Related
I am building a frontend application using react and I am wondering whether it is risky to expose API endpoint URLs (ex: https://myapi.com/v1/getitems) on GitHub, assuming that the endpoint has several security measures that are not exposed such as CORS and JWT Token Bearer Authentication. I would assume not, since, if someone were to take the endpoint and send requests, they would need a token and be allowed to do so by CORS.
Is there any security risk in doing so?
Yes. Don't add the base url of your api on github in plain view. Even though you might have CORS and Authorization, that doesn't stop a malicious actor to keep spamming your back-end.
What you do is create a .env file in your root folder. You can use a library like #beam-australia/react-env and there are others as well.
You add in the .env file the values that are important for your environment and that are usually secrets, and you want them to not be visible in your code. You do it like so:
API_URL="https://myapi.com/v1"
And then you access this variable in your code with env("API-URL") ( in the #beam-australia/react-env case, but others libraries work the same). Keep in mind that you need to add .env in .gitignore so that the .env file is not pushed to github.
Regarding requests, you can make one like so:
fetch(`${env("API_URL}/getitems`)
This way your code will be stripped of the API's base url and if someone sees your code it will see only the endpoint, not the full url.
Publishing the code of the API is risky on its own. Somebody can find a vulnerability in it and instantly hack it. If you add the address of the API to the code you help this kind of attacks. They can get the address with some investigation; OSINT and social engineering too, but better to reduce the attack surface.
As of the secrets, they must never be near to the code, because you or another developer can accidentally publish it. It happened too many times with many developers, so better to take this seriously. If you want to keep the address in secret, then you must extract it from the code and put it in the configuration of the production environment which is imported from a totally different location than your code. Using environment variables like Alex suggested is a good idea for this. Nowadays it is common to use docker, which has a way to manage secrets, so you don't need to reinvent the wheel: https://docs.docker.com/engine/swarm/secrets/ Another aspect that the configuration belongs to the actual instance. In OOP terms you want to describe the injected properties of an object in its class or in a global variable, which is an antipattern.
As of client side REST clients like javascript applications running in the browser or Android/iOS apps, you must not publish your secrets along with the REST client, otherwise it is not a secret anymore. In that case you need a server side part for the REST client and for example sign and encrypt JWT there with a secret key. It is your decision whether this server side part of the REST client sends the HTTP request to the REST API and in that case you can hide the URI of the REST API or it just manages the JWT and the client side part of the REST client sends it. If the server side part of the REST client sends the HTTP request to the REST API, then you can even use traditional sessions with session cookies between the client side and the server side parts of the REST client, but you cannot use them between the server side part of the REST client and the REST API where the communication must be stateless. Though it does not make much sense to have a separate REST API if you don't have multiple different REST clients in this scenario e.g. browser clients for JS and JSless browsers, Android and iOS clients, fully automated clients running on servers, etc. So don't confuse the REST client - REST API relationship with the browser - HTTP server relationship, because they are not necessarily the same. Most of the REST clients run on servers, not in the browser.
I've been studying logs in Logs Explorer in Google Cloud Platform and noticed that available logs don't show the body of a POST request, which is crucial for my task (I run Google Tag Manager Server-Side and need to study logs of all inbound Measurement Protocol requests). Is there a way to enable the body contents for requests like this one?
There's no built in feature in GCP Logging to do this just as norbjd said.
Only thing that's being logged by default are GET requests and only the first line:
First line of the request, containing method, path, and HTTP version. Example: GET / HTTP/1.1
Have a look at the answer here and then the fallowing comments fallowing:
A logging.info() shows up (link)
I have, and it does show up (with an i left to it) in the Stackdriver request logs and that is the only way to see POST arguments (link)
You may also try a solution from the next answer:
post request data isn't shown in request logs.
In your app, you must explicitly log it
logging.log(myRequest.PostData)
Edit: just be careful logging sensitive customer information.
I'm trying to write a VSCode extension where users could log into Google AppEngine with a google account, and I need to get their SACSID cookie to make appengine requests.
So I'm opening a browser window at
https://accounts.google.com/ServiceLogin?service=ah&passive=true&continue=https://appengine.google.com/_ah/conflogin%3Fcontinue%3Dhttp://localhost:3000/
(generated by google.appengine.api.users.create_login_url)
The user logs in and is redirected to my local webserver at
localhost:3000/_ah/conflogin/?state={state}
Now I try to forward the request to my AppEngine app (since it knows how to decode the state parameter), so I do a request to
https://my-app.appspot.com/_ah/conflogin/?state={state}
basically just replacing localhost with the actual app.
but it doesn't work, presumably because the domain is different. I assume this is on purpose, for security.
Is there any way I can make this work ?
Not ideal, but the only solution I've found is to have an endpoint on my GAE instance that does the redirection. Then I can set that as the continue url, when I'm starting the authentication process
https://accounts.google.com/ServiceLogin?service=ah&passive=true&continue=https://appengine.google.com/_ah/conflogin%3Fcontinue%3Dhttps://my-app.appspot.com/redirect?to=http://localhost:3000
I think you should center the attention on the protocols you are using, since it’s known that the cookie name is based on the http protocol (HTTP : ACSID, HTTPS:SACSID), and that’s the security perspective till this point for me.
Having the error you are facing now would be helpful to understand the problem better. Also, how are you performing the call to the API and the code you are using would be helpful too.
I am trying to implement a REST protocol and have realized in trying to debug that my web server is disallowing the PUT request.
I have tested and confirmed this further by running:
curl -X PUT http://www.mywebserver.com/testpage
Which for my web server gives back a 403 - Forbidden error.
The same happens for DELETE, where as for POST and GET everything is fine.
I am wondering if this is a common issue that those who use REST encounter and what the work-around might be?
Could I make a simple change to an .htaccess file? Or do I need to modify the protocol to set a hidden variable "_method" in the POST query string?
Often web servers will be configured to block anything except GET and POST since
99% of the time they're all that are needed and there have been problems in the
past with applications assuming the requests were one of those two.
You don't say which server it is but, for example, you can tell Apache which
methods to allow with the directive:
eg:
<Limit POST PUT DELETE>
Require valid-user
</Limit>
It sounds like maybe some helpful sysadmin has used this to block non GET/POST
You could try an .htaccess with
<Limit GET POST PUT DELETE>
Allow from all
</Limit>
(I'm not an expert at apache, this may not be exactly correct)
I have the oAuth stuff working as expected. No challenges there. I now need to use the access token to do something. Nothing I have tried works. I'm sure it's very simple, but I am not sure how to translate the examples in curl to http post/get requests.
Box.com help says:
curl https://api.box.com/2.0/folders/FOLDER_ID \
-H "Authorization: Bearer YOUR_ACCESS_TOKEN"
How do i write that using http post/get?
As long as the requests are being sent via standard http functionality I don't believe my platform matters. Regardless, I'm using Apex to write this in Salesforce.com.
Note: I know there is an app on the AppExchange to integrate Box.com and Salesforce. For my purposes I don't want to rely on apps that are unique to a specific platform.
Any help is appreciated.
It's simply a GET request with headers and optionally XML / JSON as POST payload, not sure what's your specific problem? Have you actually tried writing some code?
Simple example: http://www.salesforce.com/us/developer/docs/api_asynch/Content/asynch_api_quickstart_login.htm
Bit more advanced: http://developer.force.com/cookbook/recipe/scheduling-an-apex-call-from-the-command-line
CURL is a powerful tool and the list of it's commandline options is not for the faint of heart. Still - use it as a reference, especially "-H" meaning HTTP header. I think the "\" is just indicating that it's a multiline command, you can ignore it.