If a subscribing node in DolphinDB crashes, is there a way to automatically resubscribe in DolphinDB?
I wonder if there is a function like #reboot in cron that can enable automatic resubscription after the database restarts.
If you need automatic reconnect, specify the parameter 'reconnect' in function 'subscribeTable'.
Related
I'm trying to do one action after the flink job is finished (make some change in DB). I want to do it in the same flink application with no luck.
I found that there is JobStatusListener that is notified in ExecutionGraph about changed state but I cannot find how I can get this ExecutionGraph to register my listener.
I've tried to completely replace ExecutionGraph in my project (yes, bad approach but...) but as soon as it is runtime library it is not called at all in distributed mode, only in local run.
I have next flink application in short:
DataSource.output(RichOutputFormat.class)
ExecutionEnvironment.getExecutionEnvironment().execute()
Can please anybody help?
In my program I have enabled checkpointing,
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(5000);
but I haven't configured any StateBackend.
Where is the checkpointed state stored? Can I somehow inspect this data?
The default state backend keeps the working state on the heaps of the various task managers, and backs that up to the job manager heap. This is the so-called MemoryStateBackend.
There's no API for directly accessing the data stored in the state backend. You can simulate a task manager failure and observe that the state is restored. And you can instead trigger a savepoint if you wish to externalize the state, though there is no tooling for directly inspecting these savepoints.
It's not the answer, but small addition to the correct answer. I can't write comments cause of reputation.
If you use flink version earlier then v1.5 then default state backend will MemoryStateBeckend with asynchronous snapshots set to false. So you will use synchronous saving checkpoints every 5 seconds in your case (your pipeline will block every 5 seconds for saving checkpoint).
To avoid this use explicit constructor:
env.setStateBackend(new MemoryStateBackend(maxStateSize, true));
Since flink version v1.5.0 MemoryStateBackend uses asynchronous snapshots by default.
For more information see flink_v1.4 docs
Is there a way to automatically backup EventStore, didn't find any solution, other than manual scripts/console apps etc. I am using Get EventStore.
Unfortunately, as of right now I believe your best bet is to either:
Create an additional node in a separate environment which asynchronously replicates - this acts as a sort of 'hot backup', however this does NOT protect against data destruction via application or user command (unless you catch it before synchronization).
Roll your own script to create copies of the *.chk and other necessary files and drop them off at a secure backup location.
Outside of that, I don't know of any automated/native backup functionality in EventStore.
Here is a link to the current (albeit limited) EventStore backup documentation: http://docs.geteventstore.com/server/3.1.0/database-backup/
It's actually all in the subject: When using Go's very convenient delay package, can I still upload a queue.yaml file and configure the limits etc. for the "built-in" queue? If so, is the queue name simply "delay"? Do any other restrictions apply?
Haven't tried this myself, but this page says:
The default queue is preconfigured with a throughput rate of 5 task
invocations per second. If you want to change the preconfigured
settings, simply define a queue named default in queue.yaml.
As the subject suggests I'm interested in triggering Jenkins on changes involving a pre-configured database table. For example, whenever the number of records changes I want Jenkins to perform some particular action. Is there a plugin out of the box available for this scenario?
Thank you!
Regards,
Alex
Either you have a command line client for your database or you can write a script (perl, ruby, Groovy, Java whatever) to get this functionality. This script can be executed by Jenkins. Based on the absence of information about which database we are talking about i can't give you a more detailed hint.
What database are you using?
Most of them have some kind of triggers that can be fired after table insert, update or delete.
A logical alternative to database triggers is polling: write a script that will poll the database and store the results you are watching. If they change the script can modify a file which will trigger a Jenkins build via FS Trigger plugin.
Probably the easiest way is to use a ScriptTrigger, which can easily use embedded Groovy or Shell/Windows Batch script to pool database with a query to verify state of the given data.