I'm trying to diagnose some performance issues, so I have a the Datomic transactor running locally backed by a local instance of DynamoDB. What I can't figure out is how to populate it from a backup of our primary Datomic environment. I know the basic command is:
>datomic restore-db s3://<BUCKET> datomic:ddb://<REGION>/<DB-NAME>
but how to I tell datomic to use the local dynamodb? It seems to only accept the valid AWS regions for REGION. I've also tried using datomic:ddb-local as the protocol but no luck there either.
How do I form the target URI? Or is this even possible?
You should be able to use a ddb-local URI as indicated here: http://docs.datomic.com/storage.html#dynamodb-local
It will be something like: datomic:ddb-local://localhost:8000/my-table/my-db-name?aws_access_key_id=ABC&aws_secret_key=DEF, assuming you're running ddb-local at localhost on port 8000.
Note that the ddb-local protocol does require an access key and secret, even though they are ignored.
Best,
Marshall
Related
I'm trying to connect an application using SolrJ to a SolrCloud cluster via ZooKeeper which is secured using ACLs.
No problem you say? Right, we can just use the specific Java Properties (e.g. -DzkACLProvider etc.) and be done with it. Unfortunately we do not want to set these options with system properties because they would also contain the password. Our preferred way would be to provide these flags via environment variables. Looking into the SolrJ code this looks impossible without implementing some wrappers/extensions of SolrJ classes.
I've come so far that I think that maybe we're just doing it wrong, especially because I don't find any questions regarding this. Is this even the right way to do this? Should we even connect via Zookeeper? Or would you normally just connect to a loadbalanced SolrCloud HTTP endpoint and not connect to Zookeeper at all?
I'm thankful for any input!
i am using wso2 api-manager 02.01.00 on a linux system. The Api-Manager is deployed at Folder A. The Databases (h2) are deployed ad Folder B which is not in Folder A. The datasources in /repository/conf/datasources/master-datasources.xml are pointing correctly to the databases in Folder B. I configured it like that, because i want do preserve the databases if there is a deployment. (Becaus a fiew Developer are using the API-Manager and they don't want to loose their Data.) But it seem, that WSO2AM_DB.h2.db is created new if there is an api-manager-depoyment. I think this, because i had a look to the DB-Size. I started with a Size of 1750KB for WSO2AM_DB.h2.db. I published a view API's in the Manager and the Size increases to 2774KB. Then i did a Deployment and the size returned to 1750KB.
Effect is that API-Store/Publisher says "There are no APIS published yet".
But i could see the APIS at Application Subscriptions and in Carbon Resources at /_system/governance/apimgt/applicationdata/provider/admin.
I tried to force a new Indexing with this, but it doesn't change anything.
Could i configure at any place, that the Database should not be created/manipulated at start?
Meanwhile i'm really desperated of not solving this problem.
Maybe you could help me.
Thank you for your Time.
WSO2 does not recommend to run on H2 database. You need to use a production database such as mysql, oracle, etc. H2 is only for tryouts.
Basically, WSO2 servers store data in databases as well as use the file system. For this kind of a deployment, you need to do the following.
Point to an external database. If you are using this for demo purposes, still you can go with the current mode (H2 database).
Use dep-sync. The content which comes under the WSO2_HOME/repository/deployment/server location needs to be preserved. You can use SVN based dep-sync or rsync. Basic idea is that for a new deployment, you need to have the data of the previous deployment.
Solr Indexing preservation. If you have hundreds/thousands of APIs in the system, it would take time for indexing. To avoid that you can copy the content of WSO2_HOME/solr to the new deployment.
I have a database (mysql) at localhost. I use Solr to index and make queries, it works fine.
Now I want to put the Solr index on a remote host. I know that I must have the permission to run java and ssh access. But I must admit that, even though I can make Solr work, I don't understand very well each file that are part of Solr.
So what I want to know is which are the files estrictly necessary to make Solr index work to make queries? data files? yes, but what else? And how should I configure the permissions? read only? execute? And... I guess that there are another security items I must pay attention.
Usually you have to install at least a web-container (e.g. Tomcat or Jetty) and deploy the solr-[version].war file onto it.
For security reasons you can restrict the access to the Server via web-interface in the configuration of this web-container.
Beneath that solr needs a home-directory in which Solr stores its index and configuration. I think this must have rw permissions since Solr changes the index on import and maintains index-usage-information.
I've got a SOLR instance running behind a firewall. I'm about to put up another instance which will not be firewalled. Howevever, SOLR appears to only support pull replication and not push replication.
What are my options with regard to maintaining the same level of security? I'd rather not open too many ports in the firewall. Would HTTP over a SSH tunnel be the best option? Would it also be possible to just replicate the index files using plain old rsync (not using any SOLR specific features) or would this break something?
Would it also be possible to just replicate the index files using plain old rsync
Solr actually supports this kind of distribution with its snappuller mechanism, documented here: http://wiki.apache.org/solr/CollectionDistribution
I would open a port and specify the IP address of the slave, and just use ordinary HTTP-based replication; that would be quite secure, I think, and easier to maintain probably. I know it's not exactly where you were angling, but it's what I'd recommend.
I'm answering my own question as the solution i went for is different than what the two other answers suggested. I ended up using a SSH tunnel for HTTP traffic. Thus, i used SSH to redirect all traffic to port 8080 on the HostA to port 8080 on hostB through a SSH tunnel.
The solution appears to be working fine. I'm using a script which validates the tunnel every 5 minutes or so.
You could use HTTP basic authentication (see https://wiki.apache.org/solr/SolrReplication#Slave) but since the password will be passed in plain text, an SSH tunnel or secure VPN would also be required in order to deter more determined attackers.
I'll be going for a VPN solution to start with and consider an SSH tunnel before moving to production if we feel we are unable to place sufficient trust in our internal networks.
Is there a simple way to determine what database is in use behind a website from an external HTTP request? i.e., I make an HTTP request, get back whatever data is going to come from the webserver - can I inspect any of that and reliably determine that DB in use? I am thinking not, but figured I would ask this group.
No. The same answer could come from a static file, a SQL database, or a martian telepath.
No and for a good reason. If there were it would be a security hole. Unless it is a part of the application functionality.
For most websites, the answer is no, however, you may find security holes which reveal this information. For example, it's possible to get this information if the site isn't coded against sql injection attacks. For example, try entering the following as your user name:
'; select version();
On shared hosting system, they often don't have a firewall protecting the database from external connections.
Try the following:
telnet localhost 3306
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
5
5.0.51a—Bjb-W
This tells you that the server is running MySQL version 5.0.51a. MSSQL and Sybase also identify their version number before the client attempts to login.
Probably the easiest way is just to ask the webmaster. If your not a hacker, and the site isn't a bank, they will likely tell you.