Recently I upgraded my zeppelin from 0.8.1 to 0.9.0-preview (Also upgraded spark from 2.2 to 3.0,1).
Here I am not able to execute notebooks parallelly(by same user or different user). First executed notebook submit job on spark keeps running on other hand all other notebooks shows as waiting.
Even after first notbook is successfully completed, other notbooks not able to execute.
I was able run multiple notebook simultaneously in previous version.
Setting in zeppelin intrpreter is
You get only one session when you share your session globally. That means every executed paragraph is queued and processed sequentially as shown in the picture below:
Depending on your working environment you should change your setting to per note or to per per user (in case of a multiuser environment) and scoped or isolated mode.
Below is an overview from the official documentation of the advantages and disadvantages of the shared, isolated, and scoped mode from a notebook perspective:
Related
I try all the possible configuration in ClickOnce but the application only updates on restart.
I want the application to update after the application startup but in VS 2022 the update after startup is blocked as we can see in the following image:
I am using .net core 6 and my IDE is VS2022.
According to the documentations, checking for update before the application starts, is the default and only option available for .NET 3.1 and newer versions:
For .NET 3.1 and newer applications, checking updates before the application starts is the only update option supported.
ClickOnce update strategies
Here is a summary of main ClickOnce update strategies:
Automatic update - Check for updates after application startup
In this strategy, ClickOnce will check updates in the background when the application starts. Then, the next time when the user starts the application, based on the result of previous check-for-update, if an update was available, it shows a prompt to the user asking to download and install the updates.
Main use-case for this scenario is low-bandwidth networks or large applications when updates takes longer to download and install.
Automatic update - Check for updates before application startup
In this strategy, every time the user starts the application, the application first checks for an update and if an update is available, it downloads and applies the updates and then start the application.
It's the default strategy and as I mentioned above, it's the only available option for .NET 3.1 and above.
Main use-case for this scenario is high-bandwidth networks and also cases which you want to make the update mandatory.
Manual update - Provide a user interface for updates
In this strategy, developer writes code using the ApplicationDeployment class to check for updates; For example checking for update based on when users clicks on a menu item or button; or based on a custom user-defined schedule.
The main usage scenario for this strategy is when you want to have different strategies for different users, or custom scheduled updates for different users, or to have a manual check for update button/menu.
To learn more, take a look at the following doc articles:
Choose a ClickOnce update strategy
How ClickOnce performs application updates
How to: Check for application updates programmatically using the ClickOnce deployment API
Recently, I encountered a problem in Flink Logging in Standalone cluster mode when using logback.xml as logging. My requirement is that all my jobs should log in the particular folder and my flink framework logs should be placed in the seperate folder and also for each job running in my flink cluster there should be seperate folder for different jobs. I tested it in my local cluster which works fine and i get all my logs seperate folders respective to my Flink job submitted but as soon as i deploy my code in the Standalone cluster along with respective logback.xml for each job it doesn't logs at all. I also referred the follow. link for my query but still i am stuck with the problem.
Flink logging limitation: How to pass logging configuration to a flink job
Could you please specify where your log file resides ?
According to flink docs, it should either be specified explicitly by setting the environment property -Dlogback.configurationFile=<file> or by putting logback.xml in the classpath - usually, I overridden the one in flink/conf directory.
My team are trying set-up Apache Flink (v1.4) cluster on Mesos/Marathon. We are using the docker image provided by mesosphere. It works really well!
Because of a new requirement, the task managers have to launched with extend runtime privileges. We can easily enable this runtime privileges for the app manager via the Marathon web UI. However, we cannot find a way to enable the privileges for task managers.
In Apache Spark, we can set spark.mesos.executor.docker.parameters privileged=true in Spark's configuration file. Therefore, Spark can pass this parameter to docker run command. I am wondering if Apache Flink allow us to pass a custom parameter to docker run when launching task managers. If not, how can we start task managers with extended runtime privileges?
Thanks
There is a new parameter mesos.resourcemanager.tasks.container.docker.parameters introduced in this commit which will allow passing arbitrary parameters to Docker.
Unfortunately, this is not possible as of right now (or only for the framework scheduler as Tobi pointed out).
I went ahead and created a Jira for this feature so you can keep track/add details/contribute it yourself: https://issues.apache.org/jira/browse/FLINK-8490
You should be able to tweak the setting for the parameters in the ContainerInfo of https://github.com/mesoshq/flink-framework/blob/master/index.js to support this. I’ll eventually update the Flink version in the Docker image...
I was working on the tensorflow object detection API. I managed to train it locally on my computer and get decent results. However, when I tried to replicate the same on GCP, I had several errors. So, basically, I followed the documentation mentioned in the official tensorflow -running on cloud documentation
So this is how the bucket is laid out:
Bucket
weeddetectin-data
Train-packages
This is how I ran the training and evaluation job:
Running a multiworker training job
Running an evaluation job on cloud
I then used the following command to monitor on tensoboard:
tensorboard --logdir=gs://weeddetection --port=8080
I opened the dashboard using the preview feature in the console. But it says no dashboards are active for the current data set.
No Dashboards are active
So, I checked on my activity page to really see if the training and evaluation job were submitted:
Training Job
Evaluation Job
It seems as if there are no events files being written to your bucket.
The root cause could be that the manual your are using refers to an old version of the tensor models.
Please try and change
--train_dir=gs:...
to
--model_dir=gs://${YOUR_BUCKET_NAME}/model
And resend the job, once the job is running check the model_dir in the bucket to see if the files are written there.
Check out: gcloud ml-engine jobs documentation for additional read.
Hope it help!
Is there any utility that will copy the "official" build of a windows forms app from a central network share and launch it (from a client desktop)? I want to make sure users get the latest version when I update the binaries on the central network share.
ClickOnce is user un-friendly so I'm looking for something else...
Is it possible you could revise your question to describe what it is you find unfriendly about ClickOnce? In my office we have found ClickOnce to be the most efficient and user-friendly way of updating and distributing applications desktop business apps that we have ever had. I'm wondering if the best way to resolve your question might be to address the issues you have with ClickOnce, rather than integrating/rolling another solution.
I've done this before by the following method:
1 - Keep the "official" build at a specific network location
2 - User launches program from their local machine
3 - At launch, program compares its' own file version # to the one on the server.
4 - If the two versions are different, copy the new version down from the server and relaunch.
Pretty simple, and it works as long as you are in an intranet environment.
Step 4 is the only tricky part. You can't replace a file while it's in use, so you have to either
1 - first rename the current (in-use) file and then copy down the new one. Since you will be updating many times, you'll also want to delete any existing renamed copies that are hanging around.
or
2 - Have the user launch a "helper" application that does the version check, updates if necessary, and then launches the real app. Of course then you have to deal with updating the helper app.
We have a tool that would do that, which has been in use before there was such a thing as Windows Update (or any other update.)
The problem with any sort of update of this fashion is the security level of the user. Many times you need to be administrator to perform certain functions.
Our solution is two part/one executable: 1. a service mode that runs local system or admin to perform such operations. 2. an executable which can be called by an app to fetch via UNC, HTTP, FTP the updates for an application and apply them.
The basic process is this:
1. Application checks its version number; we use a central database to list all applications and their version numbers.
2. If the application is a minor revision we give the user an opt out on the install; if it is a major revision we require an install.
3. Once the update is confirmed, we call the updater executable which in concert with its service mode product, retrieves the updates, installs them, and relaunches the application.
If you are interested, go to the website listed in my profile and send us a support request address to me and I will give you more details and the codebase if desired.
Check out this one:
.NET Client Applications: .NET Application Updater Component
It is a white paper which discuses in detail on what it takes to make an application auto-updatable.