I have an optimization code running in sagemaker. I want to run the code in every 1 hr. How can I schedule the run in sagemaker?. I do not want to call model endpoint, but I want to run the whole code in every 1 hr.
There are couple of ways you can schedule to run training job
Call create-training-job API from a scheduled Lambda function or a CloudWatch Event .
Trigger Step Functions from CloudWatch Event to schedule tasks in SageMaker
Related
I wanna run flink programs on demand submit them went one conditions happens. How run flink jobs from java code in flink 1.3.0 version?
You can use Flink's REST API to submit a job from another running Flink job. For more details see the REST API documentation.
I have a Java application to lunch a flink job to process Kafka streaming.
The application is pending here at the job submission at flinkEnv.execute("flink job name") since the job is running forever for streamings incoming from kafka.
In this case, how can I get job id returned from the execution? I see the jobid is printing in the console. Just wonder, how to get jobid is this case without flinkEnv.execute returning yet.
How I can cancel a flink job given job name from remote server in Java?
As far as I know there is currently no nice programmatic way to control Flink. But since Flink is written in Java everything you can do with the console can also be done with internal class org.apache.flink.client.CliFrontend which is invoked by the console scripts.
An alternative would be using the REST API of the Flink JobManager.
you can use rest api to consume flink job process.
check below link: https://ci.apache.org/projects/flink/flink-docs-master/monitoring/rest_api.html.
maybe you can try to request http://host:port/jobs/overview to get all job's message that contains job's name and job's id. Such as
{"jobs":[{"jid":"d6e7b76f728d6d3715bd1b95883f8465","name":"Flink Streaming Job","state":"RUNNING","start-time":1628502261163,"end-time":-1,"duration":494208,"last-modification":1628502353963,"tasks":{"total":6,"created":0,"scheduled":0,"deploying":0,"running":6,"finished":0,"canceling":0,"canceled":0,"failed":0,"reconciling":0,"initializing":0}}]}
I really hope this will help you.
I've been using build pipeline plugin with Jenkins (v1.534) for a long time now and recently I've tried to create a pipeline with the same job (using different parameters) twice and it seems not possible. It looks like this:
Job A (param env=dev) -> Job B -> Job A (param env=qa)
Is this possible using build pipeline plugin (v1.4)?
You can try the Jenkins FLOW plugin... https://wiki.jenkins-ci.org/display/JENKINS/FLOW+Plugin
I think this is only possible if you have Job B automatically setup to trigger job A again, not manual build step.
Job B will automatically trigger downstream via the parameter trigger job plugin. This works fine if you use the automatic build downstream, but the manual hold feature post build job is not smart enough yet to continue on.
Jenkins also has massive failing in plucking upstream variables into downstream jobs, like if jobs are run out of order on a pipeline.
At my work, I duplicate jobs and chain them Build->Deploy to Dev->Deploy to QA->Deploy....XXX and so forth.
I tried to delete one kind of entity at once from GAE datastore admin page. The problem is, I fired two jobs for deleting one kind (same kind). After one job successfully finished, another just freeze, preventing other jobs from being run.
The job description is:
Job #158264924762856ED17CF
Overview
Running
Elapsed time: 00:00:00
Start time: Tue Nov 20 2012 09:58:27 GMT+0800
entity_kind: "CacheObj"
Counters
How can I clear these jobs? Deleting them from task queue won't help much, they are still inside datastore admin page.
I faced the same problem, but the frozen job didn't prevent new jobs to be executed. However, getting a frozen job is misleading. My workaround:
Go to the Datastore Viewer
Select _AE_DatastoreAdmin_Operation as kind
Find the frozen job
Delete it
You might get an error saying that the app failed
Go back to the Datastore Admin, and check that the job is no longer there
Recently I've started using limited staging on my Google App Engine project. The data is still shared between all versions, but behaviour (especially user facing behaviour) is different.
Naturally when I implement something incredibly new it only runs on the latest version of my code and I don't feel like it should be backported to the older versions.
Some of this new functionality requires cron jobs to be run periodically, but I'm hitting a problem. I have to run a cron job to call the latest code, but this is what Google's documentation has to say about the issue:
Cron requests are always sent to the default version of the application.
The default version is the oldest because the first versions of the client code that went out to users weren't future proof and don't know how to select which API version to call.
So my question is, how can I get around this limitation and make a cron job that will call the latest rather than the default version of the application?
You can now specify a version using the target tag.
<target>version-2</target>
You can't change the cron jobs to run on a different version then the default.
Depending on how much time your cron job takes to run you could change your cron job script to to do a URLFetch to "http://latest.appname.appspot.com/cron_job_endpoint".
If you're cron job takes longer then 10 minutes to run, then I would design it in a way that you can chain the different tasks using task queues.