In local mode, can I start another task manager?
Although I tried ./taskmanager.sh start, the number of task manager
at web dashboard didn't change.
The command ./taskmanager.sh start -m localhost:6123 didn't work also.
What should I do?
Is it impossible to start multiple task manager in local mode?
To start another taskmanager, you should run the following inside the flink binary directory:
bin/taskmanager.sh start
Which should update the number of taskmanagers on the web dashboard and give you an output like this:
[INFO] 1 instance(s) of taskmanager are already running on my-localhost.
Starting taskmanager daemon on host my-localhost.
From my understanding, you want to set up a standalone cluster in your
local machine. If that is the case, you could simply edit the
$FLINK_DIST/conf/workers, in which each line represents a TM host. By
default, there is only one TM in localhost. In your case, you could
add a line 'localhost' to it. Then, execute the
$FLINK_DIST/bin/start-cluster.sh, you could see a standalone cluster
with two TM in your local machine.
https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/cluster_setup.html#configuring-flink
This option wont work for windows.
https://lists.apache.org/thread.html/r7693d0c06ac5ced9a34597c662bcf37b34ef8e799c32cc0edee373b2%40%3Cdev.flink.apache.org%3E
Related
I have used wildfly-maven-plugin:1.2.1.Final and deploy,undeploy & shutdown all work for locally and also remotely.
The problem I is that the start and run goals do not work for a remote host. It seems like it does not use the same config for goals like deploy, undeploy & shutdown
Any ideas how I configure so that run or start work for remote host?
Thanks,
daslan
The wildfly-maven-plugin cannot start a remote WildFly instance. With both the start and run goals the plugin starts a new process locally. There is no way, at least that I'm aware of, to start a new process on a remote machine.
The deploy, redeploy, undeploy and shutdown goals work because they send requests over the management API.
Helo,
I run a 2 machine setup with 5 Zookeeper instances on it. I know that normally minimum 3 machines are required to run a smal zookeeper quorum but for now I need to start with this 2 machines. Now I want to create a script which autostarts all the zookeeper instances automaically in case of crashes or reboots. After all I want to build a stable environment which recovers automatically the following services:
solr
solrcloud
zookeeper
shardallocation
Does somebody have any experience with this?
You require a good monitoring system for this. A simpler solution would be to write a cron jobs for all these boxes. These cron jobs would run curl or wget comands and check the output. If the output of the command is not as expected, restart your services. Also add the services to your startup with /etc/init.d so the services start with the reboot.
I want to copy some files from a network shared drive (mounted at my local machine as drive Z). I have written a Batch file to copy the contents of Z drive into my local drive. This batch file runs successfully on cmd, but i am having issue when i trigger it through Jenkins. The Jenkins gives the following error:
"The system cannot find the specified drive"
Any help regarding this, will be greatly appreciated.
Thanks,
Nouman.
If you don't want to use Jenkins-plugins or schedule-Tasks here is a "groovy" way:
By Hand:
You can use the Groovy Script-Console provided by Jenkins>Manage Jenkins>Script Console and execute the command to map the network-drive within the Jenkins-service. (Must be repeated, once the Jenkins-service is stopped)
Automation:
Write your Groovy commands to a file named "init.groovy" and place it in your JENKINS_HOME-directory. So the network-drive gets mapped on Jenkins-startup.
Groovy Commands - Windows:
Check available network drives using the Script-Console:
println "net use".execute().getText()
Your init.groovy would look like this:
def mapdrive = "net use z: \\\\YOUR_REMOTE_MACHINE\\SHARED_FOLDERNAME"
mapdrive.execute()
Yes Jenkins uses different login credentials. To map a drives through Jenkins use below command in Jenkins command prompt:
Subst U: \drive\folder
then after that your queries.
You might run into permission issues. Jenkins might be executed with different user credentials; so it does not know the configured drive for the windows share. Instead of using shell scripts I suggest to use a plugin. There is a set of Publish-over plugins that allow deployments to remote systems via a couple of protocols (ssh, cfis etc). Have a look at the CFIS plugin that allows to send artifacts to a windows share. Once the plugin is configured (ie the host is specified in the Manage Jenkins section) you can add to the post build steps Send files to a windows share where you can specify which file(s) shall be sent to which location.
Had this issue where my jenkins job was unable to read files present on the network drive.
I resolved it by adding "net use" command in your pre-build step. i.e.
Open your job.
Go to Pre Steps
From the drop down, select Execute Windows Batch Command
Enter the following command:
net use E: \[server name][Folder name] "[password]" /user:"[userid]"
Click Save
Execute the job
I was able to read files from my network drive by following the steps mentioned above.
It seemed to be a one time activity as after the initial run, I had removed the batch command from my job and it seemed to remember the mapped drive command.
Try adding debugging commands to that bat file, or as separate build step, such as net use, set (pay attention to vars like like HOMEPATH and USERNAME) and plain dir Z:\.
As said in another answer, most likely reason is that Jenkins runs as SYSTEM user, which has different permissions. One way around that is, go to services (for example open Task Manager, go to Services tab in it, click the Services button at the lower right corner of that tab), find Jenkins service, open it's properties, go to "Log on" tab and set your normal user account as one that runs Jenkins.
Basically you can access your network shared drive (Z) using by servername or IP from jenkins command. Write \\192.168.x.xxx\Your_Folder instead of z:\Your_Folder.
For example:
mkdir \\192.168.x.xxx\Your_Folder
I was trying to copy files from one remote computer to other, the easy solution which worked for me is COPY iphone.exe \192.xx.xx.xx\dev(dev is the folder name on c drive in that ip address)
A similar issue showed up for us on Jenkins slaves set up on Windows Server 2008 following this documentation. The Jenkins agent failed to access the mounted network drives even after configuring the agent service with the correct user credentials.
Troubleshooting:
Jenkins could access the mounted network drives by their drive letters when connected via the JNLP agent (Launch agent via Java Web Start).
It stops recognizing the drive letters soon after we install the agent as a Windows service. Configuring the correct user credentials and restarting the agent does not help.
We could still access the drives via the command line while logged in to the machine with the above user.
Stop the agent service from services.msc and then uninstall it by running the command jenkins-slave.exe uninstall. The slave is disconnected at this point.
Reconnect the slave by launching the JNLP agent via Java Web Start. The agent can now access the network drives again.
Synopsis:
Do not install the slave agent as a Windows service to keep accessing your mounted network drives using drive letters. But this is highly unreliable as the agent might fail to restart after a machine reboot. Alternatively, see if Jenkins can access them via \\<ip_address\of\network\drive>.
In order to access your remote drive
just use the command in cmd prompt
pushd "\sharedDrive\Folder1\DestinationFolder"
mkdir FolderName
popd
pushd >> It navigates to the shared drive by creating a virtual drive..
popd >> Gets you back to the local directory
What is the simplest way to schedule a batch file to run on a remote machine using Hudson (latest and greatest version)? I was exploring the master slave setup. I created a dumb slave but I am not sure what the parameters should be so that I can trigger the batch file in the remote slave machine.
Basically, I am trying to run 2 different batch files on two different remote machines sequentially, triggered from my machine (the master). The Step by step guide on the Hudson website is a dead link. There are similar questions posted on SO but it does not quite work for me when I use the parameters they mention.
If anyone has done something similar please suggest ways to make this work.
(I know how to set up jobs, and add a step to run a batch file etc what I am having trouble configuring is doing this on a remote machine using hudson in built features)
UPDATE
Thank you all for the suggestions. Quick update on this:
What I wanted to get done is partially working, below are the steps followed to get to it -
Created new Node from Manage Nodes -> New Node -> set # of Executors as 1, Remote FS root set as '/var/hudson', set Launch method as using JNLP, set slavename and saved.
Once slave was set up (from master machine), I logged into the Slave physical machine, I downloaded the _slave.jar from http://masterserver:port/jnlpJars/slave.jar, and ran the following from command line at the download location -> java -jar _slave.jar -jnlpUrl http://masterserver:port/computer/slavename/slave-agent.jnlp. The connection was made successfully.
Checked 'Restrict where this project can be run' in the Master job configuration, and set paramater as slavename.
Checked "Add Build Step" for adding my batch job script
What I am still missing now is a way to connect to 2 slaves from one job in sequence, is that possible?
It is fairly easy and straight forward. Lets assume you already have a slave running. Then you configure the job as if you are locally on the target box. The setting for Restrict where this project can be run needs to be the node that you want to on. This is all for the job configuration.
For the slave configuration read the following pages.
Installing Hudson as a Windows service
Distributed builds
On windows I prefer to run the slave as a service and let the remote machine manage the start up and shut down of the slave. The only disadvantage with this is, you need to upgrade the client every time you update the server Just get the new client.jar from the server, after the upgrade and put it on the slave. Then restart the slave and you are done.
I had troubles using the install as a service option for the slave even though I did it as a local administrator. I used then srvany to wrap the jar into a service. Here is a blog about it. The command that you need to wrap, you will get from your Hudson server from the slave page. For all of this to work, you should set up the slave management as jnlp.
If you have an ssh server on your target machine, you can use the ssl slave settings. These work for me like a charm. I use them with my unix slaves. So far the ssl option with unix is less of an hassle, than the windows service clients.
I had some similar trouble with slave setup and wrote up this blog post - I was running on Linux rather than Windows, but hopefully this will help.
I dont know about how to use built-in hudson features for this job - but in one of my project builds, i run a batch file that in turn uses PSTools
to run the job on a remote server. I found PS tools extremely easy to use - download, unpack and run the command with the right parameters, hence opted to use this.
Friends
I have configured WebLogic cluster with 2 managed servers and set crashrecoveryenabled to 'true' in nodemanager.properties so that in case of server crash the managed servers can start automatically.The Node manager and admin server are setup as windows services so that they can start automatically on server reboot. I have 2 questions
1.How can I make sure that the managed servers will start automatically after server reboot(I know adding managed servers as windows service is one option).
2.In nodemanager.properties do I need to set startscriptenabled to true in production environments?
thanks
Setting up a service to have the managed servers start on system reboot is the preferred approach.
I always set startScriptEnabled=true in production environments. This just uses the script to start up the managed servers.
Provided crashRecoveryEnabled is set to true and you have started each of your managed servers then it will start.
You can use wlst to check if they are running (or start them) through some sort of scheduled task if you wish.
EDIT: From the Oracle Documentation 4.2.4 Configuring Node Manager to Start Managed Servers
If a Managed Server contains other Oracle Fusion Middleware products, such as Oracle SOA Suite, Oracle WebCenter Portal, or Oracle JRF, the Managed Servers environment must be configured to set the correct classpath and parameters. This environment information is provided through the start scripts, such as startWebLogic and setDomainEnv, which are located in the domain directory.
If the Managed Servers are started by Node Manager (as is the case when the servers are started by the Oracle WebLogic Server Administration Console or Fusion Middleware Control), Node Manager must be instructed to use these start scripts so that the server environments are correctly configured. Specifically, Node Manager must be started with the property StartScriptEnabled=true.
There are several ways to ensure that Node Manager starts with this property enabled. As a convenience, Oracle Fusion Middleware provides the following script, which adds the property StartScriptEnabled=true to the nodemanager.properties file:
(UNIX) ORACLE_COMMON_HOME/common/bin/setNMProps.sh.
(Windows) ORACLE_COMMON_HOME\common\bin\setNMProps.cmd
For example, on Linux, execute the setNMProps script and start Node Manager:
ORACLE_COMMON_HOME/common/bin/setNMProps.sh
MW_HOME/wlserver_n/server/bin/startNodeManager.sh
When you start Node Manager, it reads the nodemanager.properties file with the StartScriptEnabled=true property, and uses the start scripts when it subsequently starts Managed Servers. Note that you need to run the setNMProps script only once.