I am looking at using the Camel crypto tool for processing PGP data but have a requirement that the password to the keys used be either encrypted in the configuration file or be sourced from a secure server elsewhere. Is this possible without generating my own PGP processor?
Yes see the security menu on the Apache Camel web site: http://camel.apache.org/security.html
There is a section about configuration security, where you can use camel-jasypt for that: http://camel.apache.org/jasypt.html
This allows you to store encrypted usernames / passwords etc in a .properties file, and then you can refer to these properties from Camel crypto, using Camel's property placeholder: http://camel.apache.org/using-propertyplaceholder.html
Related
I am building a Camel application to read message from Confluent Kafka. The messages are in Avro format and added below route configuration to read the Avro messages using schema registry in Camel route. When I enable the valueDeserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer,
I am not getting any messages from Kafka topic. I tested the route with out schema registry and able to consume the message.
Route definition:
from("kafka:topic1?sslTruststoreLocation=<jks file>
&valueDeserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
&brokers=host1:9092,host2:9092,host3:9092
&sslKeystoreType=JKS
&groupId=grp1
&allowManualCommit=true
&consumersCount=10
&sslKeyPassword=<password>
&autoOffsetReset=earliest
&sslKeystorePassword=<password>
&securityProtocol=SSL
&sslTruststorePassword=<password>
&sslEndpointAlgorithm=HTTPS
&maxPollRecords=10
&sslTruststoreType=JKS
&sslKeystoreLocation=<keystore_path>
&autoCommitEnable=false
&additionalProperties.schema.registry.url=https://localhost:8081
&additionalProperties.basic.auth.user.info=abc:xyz
&additionalProperties.basic.auth.credentials.source=USER_INFO");
Can you please let me know, what is wrong in above configuration for schema registry. I also tried with EndPointRouteBuilder and same issue. However the producer application which is also Camel based and uses the schema registry for publishing Avro messages is working fine.
I figured out the way to configure the basic auth with Confluent schema registry. We need to configure as below
from("kafka:topic1?sslTruststoreLocation=<jks file>
&valueDeserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
&brokers=host1:9092,host2:9092,host3:9092
&sslKeystoreType=JKS
&groupId=grp1
&allowManualCommit=true
&consumersCount=10
&sslKeyPassword=<password>
&autoOffsetReset=earliest
&sslKeystorePassword=<password>
&securityProtocol=SSL
&sslTruststorePassword=<password>
&sslEndpointAlgorithm=HTTPS
&maxPollRecords=10
&sslTruststoreType=JKS
&sslKeystoreLocation=<keystore_path>
&autoCommitEnable=false
&additionalProperties.schema.registry.url=https://localhost:8081
&additional-properties[basic.auth.user.info]=abc:xyz
&additional-properties[basic.auth.credentials.source]=USER_INFO");
Note here, we need to use additional-properties for basic.auth.user.info and basic.auth.credentials.source as mentioned above.
My issue was that the schema registry password contained special characters, such +.
So I had to wrap the property in RAW as described in the documentation [1]
Given the above example, it would then result in:
&additional-properties[basic.auth.user.info]=RAW(abc:xyz+)
[1] https://camel.apache.org/manual/faq/how-do-i-configure-endpoints.html#HowdoIconfigureendpoints-Configuringparametervaluesusingrawvalues
Exist a way to encode parameters in flink-conf.yaml file?"
For example storing parameter like 'security.ssl.internal.keystore' in encrypted format.
Flink currently does not support to encrypt configuration values. If this is required, then please create a JIRA issue with this feature request.
What Flink supports is to not display secrets in the web ui if their key contains password or secret. See FLINK-14047 for more details.
In our Apache Camel project, we are consuming a rest service which requires a .jks file.
Currently we are storing .jks file in a physical location and referring to that in Camel project. But it can't be used always, as we may be having access to the Fuse Management Console only and not to the physical location accessible from management console.
Another option is to store key file within bundle, which is can't be employed because, certificate may change based on the environment.
In this scenario, what can be a better solution to store key file?
Note
One option about which I thought was, storing .jks file within fabric profile. But could n't find any way to do that. Is it possible to store a file in Fabric profile?
What about storing the .jks in a java package and reading it as a resource?
You bundle imports org.niyasc.jks and loads the file from there. The bundle need not to change between environments.
Then you write 2 bundles to provide the same package org.niyasc.jks, one with production file and one with test file.
Production env:
RestConsumerBundle + ProductionJksProviderBundle
Test env:
RestConsumerBundle + TestJksProviderBundle
Mind that deploying both of them may be possible and RestConsumerBundle will be bound to the first deployed bundle. You can eventually play with OSGi directives to give priority to one of them.
EDIT:
A more elegant solution would be creating an OSGi service which exposes the .jks as an InputStream or byte[]. You can even play with JNDI if you feel to.
From Blueprint declare the dependency as mandatory, so your bundle will not start if the service is not available.
<!-- RestConsumerBundle -->
<reference id="jksProvider"
interface="org.niyasc.jks.Provider"
availability="mandatory"/>
Storing the JKS files in the Fuse profile could be a good idea.
If you have a broker profile created, such as "mq-broker-Group.BrokerName", take a look at it via the Fuse Web Console.
You can then access the jks file as a resource in the property file, as in "truststore.file=profile:truststore.jks"
And also check the "Customizing the SSL keystore.jks and truststore.jks file" section of this chapter:
https://access.redhat.com/documentation/en-us/red_hat_jboss_fuse/6.3/html/fabric_guide/mq#MQ-BrokerConfig
It has some good pointers.
Regarding how to add files to a Fabric profile, you can store any resources under src/main/fabric8 and use the fabric8 Maven plugin. For more, see:
https://fabric8.io/gitbook/mavenPlugin.html
-Codrin
i began to refactor/ rebuild a xml based camel project to a java based one (i need to strictly separate configuration from functional stuff).
i am new to camel and so i am stumbling over the very first route, a ftp route. The ftp url and credentials are configuration but all the rest should be set in java.
at the moment the urie looks as follows:
ftp://<fromConfig>&stepwise=true&delay=1000&move=${file:name}.trans&recursive=true&binary=true&filter=#doneFilter&maxMessagesPerPoll=200&eagerMaxMessagesPerPoll=false&sorter=#pcrfSorter
So how to do this in java. especially the stuff using beans with "#".
thx in advance
The uri is the same in Java or XML DSL. Only that in XML mind you need to XML escape the & so it becomes & etc.
The # is a lookup in the registry, see more here: http://camel.apache.org/how-do-i-configure-endpoints.html
So the lookup happens in the Camel registry which can be a facade for JDNI / Spring etc. So it depends in what container you run Camel.
You can find a bit more details about Camel registry at: https://camel.apache.org/registry.html
I have some passwords that I access via a custom written java class and save the decrypted values in the Registry as String objects.(Note: we do not have access to the Enterprise Edition to leverage the Credentials Vault features.)
I can successfully save them and access them in code but I am having trouble accessing it in the flow. Specifically the Salesforce connector:
<sfdc:config name="ConfigurableSalesforceConnection" username="${sfdc.username}" password="#app.registry.get('salesforcepassword')" securityToken="${sfdc.securityToken}" doc:name="Salesforce" url="${sfdc.url}"/
Some of the Syntax I have used is:
password="#app.registry.get('salesforcepassword')"
and
app.registry['salesforcepassword']
Is this even possible or should I explore different avenues?
Try this
#[app.registry['salesforcepassword']]
Instead of dealing with the registry, you could also encrypt the credentials in your properties file and use Jasypt to decrypt them when the file gets loaded. This is transparent for the application and way easier.
Check this tutorial: http://www.jayway.com/2008/12/09/encrypting-properties-with-jasypt/