Can I specify a custom Java trust store for the Jenkins Artifactory plugin? I'm really looking for a place to add Java options, -Djavax.net.ssl.trustStore=... and -Djavax.net.ssl.trustStorePassword=....
I was able to do this successfully with the SonarQube by adding an environment variable:
SONAR_SCANNER_OPTS="-Djavax.net.ssl.trustStore=/data/.../mycompanycerts -Djavax.net.ssl.trustStorePassword=changeit"
But downstream, I'm seeing this error when publishing to Artifactory:
org.jfrog.build.util.VersionException: Error occurred while requesting version information: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Our company is setting up a Continuous Integration environment with Jenkins, BitBucket, Artifactory, and SonarQube. Each server is connected to our domain and there is one company ca certificate for the whole company domain: *.mycompany.lcl
While searching on the internet, most people recommended to add the needed certificate to the default Java key store. The infrastructure team won't let me do this as they have a certain location that they put the trust store on all our servers. Then they know where to go if there are any updates/additions to that trust store.
Currently, it's not possible. I've opened an issue, feel free to :+1: it.
Related
I'm trying to set up ServerSide Google Tag Manager, which is a service that runs on GCP App Engine.
The setup is done with a few Cloud Shell commands, which are describede here:
https://developers.google.com/tag-platform/tag-manager/server-side/script-user-guide#create_a_tagging_server_or_reconfigure_an_existing_tagging_server
However, when I did it originally (even as GCP project owner), I got this error:
I was advised that, to solve this, I needed to manually create a VPC Network called default. So I did exactly, that, with the default network settings.
When I ran the App Engine setup commands again, I got further, but ran into a new error:
ERROR: (gcloud.app.deploy) Error Response: [3] Flex operation projects/projectid/regions/europe-west1/operations/f7d7489b-f4ef-4b8b-ac26-b9b945d3ebfc error [INVALID_ARGUMENT]: An internal error occurred while processing task /app-engine-flex/insert_flex_deployment/flex_create_resources>2022-05-12T09:17:00.642Z6448.wa.0: 'subnetwork_name' must be specified for custom mode network 'default'.
I tried the setup commands with and without a defined subnetwork in the new VPC network I created. The same error persisted regardless.
I have not been able to find any solution to this online so far, so hoping that someone here will be able to assist.
Thanks in advance!
Simon
As you created your network as "custom", you need to create by yourself at least one subnet in the same region where your app was created. After that, you must configure the subnet in the app.yaml file:
network:
name: projects/PROJECT_ID/global/networks/NETWORK_NAME
subnetwork_name: SUBNETWORK_NAME
The answer to this was that the default service in the app engine (Services in the left-hand menu in the app engine console) failed to automatically create the VPC- and subnetworks.
By manually recreating the defaultapp engine service, the correct VPC networks and subnetworks were created, and the problem was solved.
I am trying to build my ios app in debug to test it i couldn't find a debug certificate so i used a development cert.
I got this in the build
The provided provisioning profile and certificate do not match. Please ensure that the provisioning profile you provide was generated using the certificate that you provide.
To skip validation, you may add the ios.validateProvisioningProfile=false build hint
Here is my actual configuration of certificates
I couldn't find any documentation about it.
Thanks in advance
I'm assuming you made these manually and didn't use the certificate wizard to make them. The P12 certificate information is also embedded into the provisioning profile so if you use a provisioning profile that was generated with a different certificate it just won't work.
You can enable that build hint in the build hints section if you want compilation to continue but it will probably just take longer to fail and will give a more obtuse answer.
A solution would be to use the certificate wizard and redo your certificates.
If you did this manually then redo everything. There are many nuances to making a P12 file and it's possible one of those was missed.
With the release of sonarqube 5.2, the decorator and batch components have been deprecated. Going through the code I found that they want us to use MeasureComputer instead.
Now, I had a plugin which needs the email settings of the sonar setup, fetch some basic user information and the assignee of each issue.
How do I access the email settings and user information?
Update
As per this JIRA ticket, the data associated to the analysis is sent to some compute engine which resides at the sonar server. This is where the issues are created, assigned e.t.c. So, the information regarding the issue will no longer be available even in some PostJob.
User finder alternative: I am now using the user web service to fetch information I require as far the author of the issue is concerned.
IssueFilter helps me lay my hands on each issue that is reported.
My purpose here is to create a custom report immediately after the analysis where a link to each issue is shown. I don't think I'll be able to achieve this using the sonar plugin.
I try to make a SP hosted on Google app engine, with a third party IdP, and I'm facing multiple problems.
I'm using Spring SAML extension for java. I was able to run the stand alone (not in GAE) demo app, from the official guide http://docs.spring.io/spring-security-saml/docs/1.0.x/reference/html/chapter-quick-start.html using as IdP idp.ssocircle.com.
Now my problem comes whey I tried to integrate this code in my GAE project. When running with GAE I can get to the phase where I'm redirected to do the login on ssocircle.com and from there when I should be redirected back to my page I'm getting this error: "Error 401 Authentication Failed: Error decoding incoming SAML message", and on local GAE server logs I can see this message
"[INFO] Dec 17, 2014 5:21:23 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry [INFO] INFO: I/O exception (java.net.SocketException) caught when processing request: Permission denied: Not allowed to issue a socket bind: permission denied."
I was wondering if this is a limitation from GAE regarding sockets: 1) Sockets are available only for paid apps (I don't have a paid app, but could consider this options) 2) You cannot create a listen socket; you can only create outbound sockets. 3) You cannot bind to specific IP addresses or ports. (for me this one sounds like it is my problem).
And whey I tried to deploy on GAE, the app remained in a start-up loop because of other errors regarding "nested exception is java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "modifyThreadGroup")", that sounds like another GAE limitation to me.
At some point I was ready to gave up, because I was thinking that SAML Spring extension can't work with GAE, because of limitations present on GAE. But I see you have the same project running as a demo on GAE (or at least I think it is running on GAE because of the appspot part in domain name). https://saml-federation.appspot.com/saml/discovery?returnIDParam=idp&entityID=saml-federation.appspot.com
I would appreciate if you can give me some hints regarding my problems, and best would be if you could help me with the source code of this demo project (could not find it anywhere), and all configuration that is needed for GAE.
I've created a new repository https://github.com/vschafer/spring-security-saml-gae which includes instructions for deployment of Spring SAML applications on Google Application Engine. It also includes classes helping to avoid issues you're facing (sockets and threads).
In order to use it:
include the jar created from the repo in your project
use the provided org.springframework...StaticFilesystemMetadataProvider for loading of your metadata
in case you are using HTTP-Artifact binding replace bean org.springframework...ArtifactResolutionProfileImpl with org.springframework...google.ArtifactResolutionProfileGAE
Please comment if you spot some mistakes in the documentation or code.
This is specifically a question relating to server to server authorisation between a python Google AppEngine app and Google's BigQuery, but could be relevant for other cloud services.
tldr; Is it possible to get the App Engine local development server to authenticate with the remote BigQuery service? Better yet is there a local BigQuery?
I understand that AppAssertionCredentials does not currently work on the local development server, though that in itself is very frustrating.
The alternative method which works for standard python code, outside of the local development server sandbox, detailed here does not work for the local development server because even with PyCrypto enabled the sandbox does not allow some posix modules e.g. 'pwd'.
I have got AppAssertionCredentials working on the remote server and the SignedJwtAssertionCredentials method working in native python locally, so the service accounts are set up properly.
The imports fail within oauth2client/crypt.py within the try/except blocks - after commenting them out the sandbox whitelist exceptions are easily seen.
I've fiddled around with adding 'pwd' to the whitelist, then another problem crops up, so I scurried back out of that rabbit hole.
I've tried including PyCrypto directly into the project with similar results.
I've also tried with OpenSSL with similar results.
I have looked for a local appengine specific PyCrypto to no avail, have I missed one? I should say this is on Mac OSX - perhaps I should fire up a linux box and give that a go?
A recent release of Google App Engine SDK added support for the AppAssertionCredentials method on the development server. To use this method locally, add the following arguments to dev_appserver.py:
$ dev_appserver.py --help
...
Application Identity:
--appidentity_email_address APPIDENTITY_EMAIL_ADDRESS
email address associated with a service account that
has a downloadable key. May be None for no local
application identity. (default: None)
--appidentity_private_key_path APPIDENTITY_PRIVATE_KEY_PATH
path to private key file associated with service
account (.pem format). Must be set if
appidentity_email_address is set. (default: None)
To use these:
In Google Developer Console, select a project then navigate to "API & auth" -> "Credentials" -> "Create new client ID".
Select "Service account" and follow the prompts to download the private key in PKCS12 (.p12) format. Take note of the email address for the service account.
Make sure you add that service account email address to the "Permissions" tab for any project that contains data it needs to access, by default it is added to the project team in which it was created.
Convert the PKCS12 format to PKCS1 format using the following command:
$ cat /path/to/xxxx-privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass:notasecret | openssl rsa > /path/to/secret.pem
Start dev_appserver.py as:
$ dev_appserver.py --appidentity_email_address xxxx#developer.gserviceaccount.com --appidentity_private_key_path /path/to/secret.pem ...
Use appidentity module and AppAssertionCredentials in the same manner locally as you normally would in production.
Please ensure that /path/to/secret.pem is outside of your application source directory so that it is not accidentally deployed as part of your application.
So searching deeper for PyCrypto and local appengine sandbox lead me onto this thread and response specifically...
https://code.google.com/p/googleappengine/issues/detail?id=1627#c22
This is fixed in 1.7.4. However, you must use easy_install -Z
(--always-unzip) to install PyCrypto. The default zipfile option in
OSX 10.8 is incompatible with the sandbox emulation in the
dev_appserver.
The solution turns out to be very straight forward...
I used:
sudo easy_install pycrypto
and it should have been:
sudo easy_install -Z pycrypto
as per the thread above. Using PIP will work as well:
pip install pycrypto
or a manual download and install of pycrypto will also work. I tested all three.
If you have installed pycrypto with easy_install and without -Z flag then you may want to install pip just so you can easily uninstall pycrypto...
easy_install pip
for the record I built and installed libgmp, as pil and the manual install showed this warning...
warning: GMP or MPIR library not found; Not building
Crypto.PublicKey._fastmath.
Although this gave me fastmath, it was not essential to solve the problem as the Crypto libs gracefully fail to slowmath.
Another point that tripped me up for a bit was I removed pycrypto from app.yaml whilst testing to see if OpenSSL might give me all I need.
So dont forget to add...
- name: pycrypto
version: latest
into app.yaml under the libraries: section.
With this missing the native _counter library was not imported hence Counter failed etc.
Also for the record any talk of having to move Crypto into the app folders themselves or out of the default Mac OS X location of /Library/Python/2.7/site-packages/Crypto was only valid in earlier versions of the dev server.
Similarly there is now no need to edit any _WHITE_LIST_C_MODULES lists (which is in sandbox.py in appengine 1.8 onwards, which also includes the regex which allows Crypto.Util._counter etc)
The other bit of the puzzle in case you get here before discovering the key issue is that the key file you download from the console is PKCS12 and is downloaded as hex text, so I converted that to binary and then converted that to a PEM so I could include it in the source code.
I struggled with this one for a day or two. And I was finally able to get localhost working with server to server authentication, a service account and a .p12 cert.
If it's at all helpful to anyone, here's a simple gist: https://gist.github.com/dandelauro/7836962
I agree with the first post - the localhost/production impedance is a real pain in the a**. AppAssertionCredentials is the right way to go on production and I don't want to have two different code paths between production and localhost. So the development environments need to be adjusted to be able to perform the required authentication without affecting the main code path.
E.g., perhaps a developer could log in with their own Google account using appcfg.py and then that auth would be cached for a period such that AppAssertionCredentials would work out. The developer's Google account could be granted permissions on the appropriate environments (dev and test for us, e.g.)
re: "local BigQuery" - we have some initial stuff in place that uses SQLLite to simulate BigQuery interactions for unit tests and other offline/local testing, but of course, it's not a great simulation. I agree that all the Cloud Platform products need to spend as much time thinking about the development-time experience as App Engine has.
Is it possible to get the App Engine local development server to authenticate with the remote BigQuery service?
I think it's impossible to use AppAssertionCredentials as authentication method between BigQuery service and your local App Engine server currently.
Alternatively, I'm using OAuth2 authentication which is associated with specific user(this user must be registered in your project at google api console) to access BigQuery from local App Engine server.
For getting user OAuth2 authentication, I use oauth2client.client module in the app code.
I hope this will be helpful to your problem.
Updated:
This is what I'm doing for getting the user OAuth2 authorization.
Edited:
Added missing import statement.
Thanks mattes!
import os
import webapp2
import httplib2
from oauth2client.client import OAuth2Credentials
from oauth2client.appengine import StorageByKeyName, CredentialsModel, OAuth2DecoratorFromClientSecrets
from google.appengine.api import users
oauth2_decorator = OAuth2DecoratorFromClientSecrets(
os.path.join(os.path.dirname(__file__), 'client_secrets.json'),
scope='https://www.googleapis.com/auth/bigquery')
oauth2_decorator._kwargs = {'approval_prompt': 'force'}
class TestPage(webapp2.RequestHandler):
#oauth2_decorator.oauth_required
def get(self):
user_id = users.get_current_user().user_id()
credentials = StorageByKeyName(CredentialsModel, user_id, 'credentials').locked_get()
http = credentials.authorize(httplib2.Http()) # now you can use this http object to access BigQuery service
application = webapp2.WSGIApplication([
('/', TestPage),
(oauth2_decorator.callback_path, oauth2_decorator.callback_handler()),
], debug=True)