Sagemaker lifecycle config: could not find conda environment conda_python3 & Any other Environment is not found - amazon-sagemaker

#!/bin/bash
set -e
ENVIRONMENT=python3
NOTEBOOK_FILE="/home/ec2-user/SageMaker/untitled.ipynb"
source activate python3
nohup jupyter nbconvert --to notebook --ExecutePreprocessor.timeout=600 --ExecutePreprocessor.kernel_name=conda_python3 --execute "$NOTEBOOK_FILE" &
The Above script which i use to start my notebook file "untitled.ipynb" when my sagemaker notebook instnace starts.
But when i start my notebook instance i am getting an error "Could not find conda environment: python3"
if any one knows the solution please post.

When you run lifecycle config, you need to specify the activate script path where anaconda resides (and typically where the environments are then).
Then just modify the code part like this:
ENVIRONMENT=python3
source /home/ec2-user/anaconda3/bin/activate "$ENVIRONMENT";
You can also see official examples of various configurations in this repo:
amazon-sagemaker-notebook-instance-lifecycle-config-samples

Related

Spyder Version 5 Standalone With Mini Conda Commands

I am using Spyder version 5 the standalone version with Miniconda.
I was hoping someone could shed some light on the following questions:-
I know how to update all the packages within a given environment, however does anyone know what the command is to update a specific package within a given environment and how to delete a specific package within a given environment?
Does anyone know what the command is to upgrade Python from say 2.7 to 2.8 using Spyder standalone within mini conda.
When I remove an environment I get the following message in mini conda:-
"Remove all packages in environment C:\ProgramData\Miniconda3\envs\minicdm:" - Whats all this about???
And finally I am trying to create a list of common commands for newbies like myself and I was wondering if anyone could add anything to the list I currently have:-
4.1 conda env list - List all environments and places an asterisk to which environment is active.
4.2 conda deactivate - Deactivates active environment, you may or may not need to be in the folder to deactivate it
4.3 conda env remove -n - Removes the environment, must be deactivated before it can be removed.
4.4 conda update -n --all - Updates all the packages within a given environment.
4.5 conda create -n -y - Creates a conda environment
4.6 conda activate - Activates a conda environment
4.7 conda install spyder-kernels scikit-learn -y - Installs a package in the activate environment
Thank you.
Do not rely on third-party information. Read the the docs:
https://docs.conda.io/projects/conda/en/latest/user-guide/index.html

VOLTTRON install on rasbian buster

Can I get a tip for installing on rasp buster? Im hung up on the install directions to check the status of the rabbitMQ server. Traceback of bash console:
(volttron) pi#raspberry:~/Desktop/volttron $ echo 'export RABBITMQ_HOME=$HOME/rabbitmq_server/rabbitmq_server-3.7.7'|sudo tee --append ~/.bashrc
export RABBITMQ_HOME=$HOME/rabbitmq_server/rabbitmq_server-3.7.7
(volttron) pi#raspberry:~/Desktop/volttron $ source ~/.bashrc
pi#raspberry:~/Desktop/volttron $ RABBITMQ_HOME/sbin/rabbitmqctl status
bash: RABBITMQ_HOME/sbin/rabbitmqctl: No such file or directory
There are a few tracebacks earlier on the installation...
If it makes a difference or not here is the entire bash console process. The git gist link I just created the name install.py even though its just bash commands copied pasted per install directions...
`pi#raspberry:~/Desktop $ git clone https://github.com/VOLTTRON/volttron --branch releases/7.x`
It looks like there are a couple of different issues going on here:
The issue you quote above (RABBITMQ_HOME/sbin/rabbitmqctl: No such file or directory) is that your shell isn't finding the rabbitmqctl command. It looks like you added the RABBITMQ_HOME environment variable to your .bashrc, but used the string RABBITMQ_HOME instead of the variable expansion $RABBITMQ_HOME when you tried to run the command. Try running it as $RABBITMQ_HOME/sbin/rabbitmqctl status instead.
The rabbitmqctl status command will check the status of the rabbitmq application, but I don't think you've done anything to start it yet (that happens when you bootstrap the platform and/or start the platform configured to use the RMQ broker)
I think that the traces earlier in the installation process are problematic (appears to be the same error hit two different ways), but you just haven't run into them yet. I haven't seen any issues building gevent on the RPi 4 with buster (though it is pretty slow), but the ctypes error makes me wonder if there's an issue with the underlying c library it is trying to build on top of. I did notice that you're getting amd64 erlang packages, are you running Raspbian on an x86 processor? (if so this isn't a permutation we've tried and you may be hitting some package compatibility edge case we haven't seen)
One thing to try is to manually install cython into your virtualenvironment and then try running the bootstrap script again with the virtualenvironment activated. You could also try and pip install gevent==20.6.1 directly in that virtualenvironment (this is what the bootstrap script was doing at the failure point). VOLTTRON depends on gevent, so if that isn't installing the platform won't be able to run.

arangodb data definition execution issue

I am trying to execute below command in arangodb kubernetes based container, where arango "Version 0.13.7, build bdac926" is running.
command which tried from my end :
arangodb-ddl-exec-cli.jar -input_file arangodb_ddl.json -db_url "http://11.22.3.5:32532" -username root -password XXXX**
while executing above command getting belwo error:
bash: arangodb-ddl-exec-cli-1.0.0-RELEASE-standalone.jar: command not found
My specification:
Docker - 18.03.1-ce
Kubernetes - v1.12.0
Kubernetes arangodb service port ( Type: Nodeport ) - 8529:32532/TCP
please let me know how to resolve it.
thanks in advance.
If I understand correctly you are trying to run a JAR file inside the container?
If yes then you should first install the JRE in the container (for example by adding it in Dockerfile or at the moment from the inside of the container).
First check if Java is installed using java --version
If it is not there - install it. When you have it run:
java -jar arangodb-ddl-exec-cli.jar and rest of your arguments.
Hope this helps.

ModuleNotFoundError in Spyder

I tried to import the biopython package in Spyder and got the error message:
ModuleNotFoundError: No module named 'biopython'
although biopython is installed.
I also checked the PYTHONPATH: there is a path set into the directory where the packages are stored.
Can somebody help? Did I miss something? Thanks for your help!
If you're using Anaconda, it's best to install all the packages you want from Anaconda if possible. You can check if a package is available with (e.g.):
conda search biopython
When I try that command it shows that biopython is available, so assuming you have access to the standard conda channels you should be able to get it this way.
Assuming you haven't already created a conda environment to work with, start by creating a new one with the packages you want to use:
conda create -n myenvname spyder biopython
where myenvname is the name you want to give the environment - call it whatever you like. If you want to use other packages as well, add their names to the end of this command. Then once the env is completed, activate it:
activate myenvname
or if this doesn't work, on Mac or Linux:
source activate myenvname
and start Spyder in this environment:
spyder
Each time you want to use this environment in future you will need to activate it first. You may also be able to do some of these tasks through the Anaconda Navigator or via Start menu shortcuts but the command line version will always work.
If there's a package you want that isn't available from conda but is available via pip, just use the pip command after creating and activating the environment.
If you are using Anaconda, a solution could be
conda install -c main biopython
following https://anaconda.org/main/biopython.
The official repository page helped me when I got your error message because numpy was not in place.

Can't find dev_appserver.py with gcloud installation

I've installed gcloud by following the instructions on:
https://cloud.google.com/sdk/docs/quickstart-debian-ubuntu
gcloud is in my path at /usr/bin/gcloud, but the package doesn't seem to have dev_appserver.py in my path. Is it installed? How do I run it?
Platform: Ubuntu 16.04
Edit: By running dpkg -L google-cloud-sdk I've found it at /usr/lib/google-cloud-sdk/bin/dev_appserver.py but when I try to run it I get:
This action requires the installation of components: [app-engine-
python]
You cannot perform this action because this Cloud SDK installation is
managed by an external package manager. If you would like to get the
Also not sure why it wasn't added to my path.
I know the original question concerns Ubuntu, but I just wanted to share some notes for macOS/OS X in case it's helpful for someone else.
I installed the google-cloud-sdk via Homebrew-Cask and overlooked the caveats note:
brew cask install google-cloud-sdk
After installing the SDK cask, I installed the Python App Engine component, as #Rodney Jonace mentioned:
gcloud components install -q app-engine-python
Going back to the caveats note mentioned above, I appended the following the my ~/.zshrc file:
source $(brew --prefix)/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/path.zsh.inc
source $(brew --prefix)/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/completion.zsh.inc
Opening a new terminal tab, I was able to call the extra Python App Engine scripts (e.g., dev_appserver.py) and use the Zsh completions. Hope that helps!
The following articles were also useful:
http://www.javatronic.fr/tips/2014/10/17/installing_google_cloud_sdk_on_ubuntu_with_oh-my-zsh.html
http://www.rainbowbreeze.it/how-to-setup-a-google-app-engine-python-environment-on-mac-osx-using-homebrew/
The google-cloud-sdk deb package comes with the built-in component manager disabled, which is preventing that copy of dev_appserver.py from working through gcloud. If you update your apt-cache, you can install the google-cloud-sdk-app-engine-python and/or google-cloud-sdk-app-engine-java packages that have just started to be published. Directions here:
https://cloud.google.com/sdk/downloads#apt-get

Resources