How do I recreate session cookies in WebDriver for IAP secured resource? - google-app-engine

I am trying to automate a site that is protected behind an Identity-Aware Proxy (IAP) on Google Cloud Platform GCP. I currently have access to a service account that I am able to make API requests with an OpenID Token.
When logging into the application normally (with username and password), I see the following cookies:
GCP_IAAP_AUTH_TOKEN_<Some GUID here>
GCP_IAP_UID
How can I use the service account credentials (available in a json file) to recreate these cookie values so that I can inject them into my selenium webdriver?

I ended up solving this using BrowserMob-Proxy. From their README:
BrowserMob Proxy allows you to manipulate HTTP requests and responses, capture HTTP content, and export performance data as a HAR file.
For Python3, I did the following:
Pre Requiements:
BrowserMob-Proxy installed and running
For Mac I used HomeBrew:
?> brew install browsermob-proxy
?> brew services start browsermob-proxy
Set up local python3 environment with pipenv (or your choice of virtual env manager)
?> brew install pipenv
?> pipenv --python 3.8
?> pipenv install browsermobproxy
?> pipenv install selenium
?> pipenv install ....
Ability to authenticate with data source for your webpage. Since I was utilizing a GCP service I followed the flow published in the IAP documentation for getting the authentication token found here: Authenticating from a service account
Simplified code for adding the proxy:
from selenium.webdriver import ChromeOptions
import browsermobproxy
# 1. Do whatever you need to do to get your token
token = get_auth_token()
# 2. Create browsermob client and add auth to headers
client = browsermobproxy.Client("localhost:9090") # port depends on your own setup
client.headers({"Authorization": "Bearer {}".format(token)})
# 3. Create browser (can vary wildly based on your own needs)
chrome_options = ChromeOptions()
chrome_options.add_argument("--ignore-certificate-errors") # I needed this, you may not
caps = chrome_options.to_capabilities()
client.add_to_capabilities(caps) # This is important!
# create driver instance with your capabilities

Related

Connecting to an external HTTP api behind a proxy from nifi

I have a apache/nifi:latest instance spun inside an Amazon Linux 2 EC2. For reference, see this guide: here
I have a QuerySalesforceObject ver. 1.18.0 that makes use of StandardOauth2AccessTokenProvider.
The oauth2 provider url is configured at https://test.salesforce.com/services/oauth2/token
I can curl this url from the box and from inside the docker container just fine (I don’t get a timeout).
[root#ip-10-229-18-107 \~\]# docker exec -it nifi_container_persistent /bin/sh
printenv | grep -i proxy
HTTPS_PROXY=http://proxy.MY_DOMAIN.com:3128
no_proxy=localhost,127.0.0.1,MY_DOMAIN.com,.amazonaws.com
NO_PROXY=localhost,127.0.0.1, MY_DOMAIN.com,.amazonaws.com
https_proxy=http://proxy.MY_DOMAIN.com:3128
http_proxy=http://proxy.MY_DOMAIN.com:3128
HTTP_PROXY=http://proxy.MY_DOMAIN.com:3128
curl https://test.salesforce.com/services/oauth2/token
{"error":"unsupported_grant_type","error_description":"grant type not supported"}#
But when I run the task, oauth2 fails with an error
java.io.UncheckedIOException: OAuth2 access token request failed
Caused by: java.net.SocketTimeoutException: connect timed out
This leads me to believe the proxy settings are not being honored by the class. How can I fix this?
Here’s more info on this class: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-oauth2-provider-nar/1.17.0/org.apache.nifi.oauth2.StandardOauth2AccessTokenProvider/index.html
The standard way to interface with HTTP resources with a proxy in Nifi is via StandardProxyConfigurationService: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-proxy-configuration-nar/1.19.1/org.apache.nifi.proxy.StandardProxyConfigurationService/index.html
If a component does not have this property, then it means it does not support it.
You can try bootstrapping proxy settings into nifi with /opt/nifi/nifi-current/conf/bootstrap.conf. But there is no standard and support of proxy is not guaranteed. Implementation (bugs and all) depends on the library. aws-java-sdk ver. 1x, for example, has a bug where nonProxyHosts is not honoured. https://github.com/aws/aws-sdk-java/issues/2797
java.arg.18=-Dhttp.nonProxyHosts="foo|localhost|*.bar.org"
java.arg.19=-Dhttp.proxyHost=proxy.foo.com
java.arg.20=-Dhttp.proxyPort=123
java.arg.21=-Dhttp.proxyUser=foo
java.arg.22=-Dhttp.proxyPassword=bar
java.arg.23=-Dhttps.nonProxyHosts="foo|localhost|*.bar.org"
java.arg.24=-Dhttps.proxyHost=proxy.foo.com
java.arg.25=-Dhttps.proxyPort=123
java.arg.26=-Dhttps.proxyUser=foo
java.arg.27=-Dhttps.proxyPassword=bar

How to setup CouchDB with JWT auth?

I'm trying to setup just for CouchDB, so I can use my current API tokens to authenticate in CouchDB.
But the docs don't seem to provide enough information to set this up.
I don't understand what the payload of the gut needs to contain identifying the user. How do I configure the jwt secret?
Is there any simple example out there or something like or tutorial to do this correctly?
this thread may be helpful:
https://github.com/apache/couchdb/discussions/2947
edit:
ok i found a configuration that worked for me - this solution won't be suitable for anything more than a testing couchdb instance.
configure local.ini (docker image)
1.1 -> ssh to your docker docker exec -it bash
1.2 -> install vim for convenience and disable visual mode:
$ apt-get update
$ apt-get install vim
$ echo "set mouse-=a" >> ~/.vimrc
1.3 -> update local.ini
$ vi /opt/couchdb/etc/local.ini
in [chttpd] section add line
authentication_handlers = {chttpd_auth, jwt_authentication_handler}, {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, default_authentication_handler}
at the very end of the file add jwt_keys config
[jwt_keys]
hmac:_default = aGVsbG8=
hmac:foo = aGVsbG8y
restart your container
configure postman :
bearer token for _default hmac is
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyXzEiLCJleHAiOjE1OTI2MTEyMDB9.Y9jNgSeSBl54V2MHg1hXhivyZsdXTeiAVJR2DSlF6LQ
put it into postman and issue following get request:
http://localhost:5984/_session
you should see something like :
{"ok":true,"userCtx":{"name":"user_1","roles":[]},"info":{"authentication_handlers":["jwt","cookie","default"],"authenticated":"jwt"}}
now go in your browser to couchdb ui and login as admin
http://127.0.0.1:5984/_utils/#login
then click the "lock" icon next to the database you wish the user_1 to have privileges and update the permissions accordingly.
check
check if your user_1 authenticating via jwt has permissions on the database by issuing appropriate request, for example
http://localhost:5984/campaigns

Openstack keystone token issue. Failed to discover available identity versions

Im attempting to setup keystone identity service on virtualized Ubuntu 20.4 via Windows Hyper-V on Windows 10.
I followed the instructions on: https://docs.openstack.org/keystone/pike/install/index-ubuntu.html very carefully & attempted this 4 times from scratch with a new VM.
However, all calls to OpenStack web service API either via curl, or command line result in some type of 404 Error.
Here my detailed steps (not including VM setup):
Install & Setup MySQL for keystone:
apt install keystone :
configure keystone:
fernet setup:
Bootstrap keystone:
Apache2 setup:
In /etc/apache2/apache2.conf
Set ServerName controller
source confif file
check port 5000.
set local host name resolution:
Start apache. service apache2 restart
Here is the issue:
Attempt # 1 - Using curl to call keystone API (404):
Attempt # 2 - Using openstack command line tool, again (404):
Troubleshooting:
Keystone logs look good with no error:
Furthermore, apache2 is running as I can see default server page:
So why cant keystone API be reached?
Appreciated
export these variables and test again
export OS_AUTH_VERSION=3
export OS_ENDPOINT_TYPE=internalURL
export OS_INTERFACE=internalURL
export OS_AUTH_TYPE=password
export OS_TENANT_NAME=admin
export OS_NO_CACHE=1
check your wsgi and apache2 vhost config too, to make sure apache will redircet it correctly
would you please check if libapache2-mod-WSGI is installed as it is mentioned in the documentation
enter image description here

mvn appengine:update from Bamboo requires credentials

I have a maven project that updates appspot with my war, but when the build is executed by Atlassian Cloud Bamboo the OAuth authentication prompts for access tokens.
14-May-2015 00:17:18 [INFO] Retrieving Google App Engine Java SDK from Maven
14-May-2015 00:17:18 [INFO] Updating Google App Engine Application
14-May-2015 00:17:18 [INFO] Running -V 1.9.20 --oauth2 update /mnt/bamboo-ebs/bamboo-agent/build-dir/[project path]-1-SNAPSHOT
14-May-2015 00:17:21 Please open the following URL in your browser:
14-May-2015 00:17:21 https://accounts.google.com/o/oauth2/auth?access_type=offline&approval_prompt=force&client_id=XXXXXXXX.apps.googleusercontent.com&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=https://www.googleapis.com/auth/appengine.admin%20https://www.googleapis.com/auth/cloud-platform
Opening the url I'm able to get an access token, but I'm not able to enter it so the build hangs until cancelled.
How should I setup authentication?
Add the oath credentials to an EBS attached volume and copy them into place when the Agent is started.
Configure bamboo for EBS attached volumes
SSH into the instance, su to bamboo user
Download the Appengine SDK
Run appcfg to be prompted for authentication
appcfg -s my-app.appspot.com version
follow instructions for oauth which will create /home/bamboo/.appcfg_oauth2_tokens_java
mkdir /mnt/bamboo-ebs/appengine
cp /home/bamboo/.appcfg_oauth2_tokens_java /mnt/bamboo-ebs/appengine
edit /mnt/bamboo-ebs/bin/customize-extras.sh (see docs) adding the following
cp /mnt/bamboo-ebs/appengine/.appcfg_oauth2_tokens_java /home/bamboo/
chown bamboo:bamboo /home/bamboo/.appcfg_oauth2_tokens_java
Take a snapshot of the volume and assign the snapshot to your bamboo instance configuration
The next time bamboo starts a new instance the customize-extras.sh will be run and the necessary credentials will be in place for mvn appengine:update to find.

Updating WordPress plugins asks for FTP credentials with GAE 1.9.18 SDK

I'm trying to update WordPress plugins running on my local GAE server but I am getting prompted to enter FTP credentials. The only thing I have changed is upgrading from AppEngine 1.9.17 SDK to 1.9.18. Does anyone know how to resolve?
I posted the issue in the App Engine issue queue and got an answer: https://code.google.com/p/googleappengine/issues/detail?id=11753
The GAE 1.9.18 SDK made local filesystem read-only by default, I had to add this line to php.ini to resolve the issue:
google_app_engine.disable_readonly_filesystem = 1
You should be able to change permissions using shh client such as Putty, Zoc etc...
and then type:
sudo chown -R www-data:www-data /var/www/folder-name/
after that it shouldn't ask you for ftp credentials.
WordPress asks for your FTP credentials when permissions aren't set correctly for WordPress to write files and folders. This can be caused by PHP running as the apache user (mod_php or CGI) rather than the user that owns your WordPress files. Upgrading may have changed the permissions of wp-content. See http://codex.wordpress.org/Changing_File_Permissions
A workaround is to put your FTP credentials directly in wp-config.php. See http://codex.wordpress.org/Editing_wp-config.php#WordPress_Upgrade_Constants
define( 'FTP_USER', 'username' );
define( 'FTP_PASS', 'password' );
define( 'FTP_HOST', 'ftp.example.org:21' );

Resources