I'd like to run an instance of Vespa outside of a container (e.g. Docker). The Docker path is definitely quite convenient and works great. But I would like to go thru the process by hand of setting up an instance on macOS and seeing more of the 'nuts and bolts' of Vespa.
It appears there are nice docs which outline a path to building RPM's for Centos, etc. Would walking thru that process and adapting to macOS be my best bet?
Unfortunately, running Vespa on MacOS directly is not yet supported. I'd suggest instead running a CentOS VM or cloud instance and experimenting there.
Related
Following https://docs.hiro.so/smart-contracts/devnet I can't get the command clarinet integrate to work. I have installed Docker on my mac and am running version 0.28.0 of clarinet. Running command within 'my-react-app/clarinet' where all clarity related files live (contracts, settings, tests, and Clarinet.toml).
My guess is it could be an issue with Docker?
The issue was that I downloaded my Devnet.toml file from a repo that was configured incorrectly. The configuration I needed was:
[network]
name = "devnet"
I increased the CPU and Memory in Docker as well.
There is an issue when the command attempts to spin up the stacks explorer, but I was informed that there are several existing issues with the stacks explorer from clarinet integrate at the moment.
Depending on how the last devnet was terminated, you could have some containers running. This issue should be fixed in the next incoming release, meanwhile, you'd need to terminate this stale containers manually.
Apart from Ludo's suggestions, I'd also look into your Docker resources. The default CPU/memory allocation should allow you to get started with Clarinet, but just in case, you could alter it to see if that makes a difference. Here's my settings for your reference:
Alternatively, to tease things out, you could reuse one of the samples (eg: hirosystems/stacks-billboard) instead of running your project. See if the sample comes up as expected; if it does, there could be something missing in your project.
My Goal
I've been using devcontainers in combination with WSL2 for a little while now. But I keep running into issues and besides that I like off-loading resources of my laptop to a server. Moving the containers to a native Linux server would solve my issues.
My ideal situation would be to have a solution that works just like working locally on my Windows laptop (later probably moving to Macbook) but using the facilities of a Linux server (which has systemd and netns) and moving the workload there as well so my laptop doesn't sound like a vacuum cleaner.
My Journey
I'm trying to setup remote containers as described here: https://code.visualstudio.com/remote/advancedcontainers/develop-remote-host
Actually the containers are running fine, I'm using the second storage solution what means I add the following to my .devcontainer.json:
"workspaceMount": "source=/home/marvink/code,target=/workspaces,type=bind,consistency=cached"
And my workflow currently looks something like this:
Clone project locally (with .devcontainer already in the project)
Add workspaceMount above to devcontainer.json
Clone project on remote (e.g. to /home/marvink/code/new-project)
Open project locally
Build and reopen in container
Work on the files on the remote
My issue
This works but now I have files on my local drive that never get touched which isn't ideal but not a disaster, the bigger issue is when I want to update the devcontainer. I need to do that locally (in a seperate window), manually need to copy paste that to the remote if I want to commit it to git and off-course I sometimes forget this and try to edit it remotely which is causing a lot of frustration (and sometimes it seems like it does use the remote config, but that might have been a mistake?).
This is why I want to setup rsync both ways to sync changes to files and as a bonus I can edit files locally when I'm offline. In the link it's described how to do it manually but I want it automated so that I can't forget or make mistakes.
From Powershell I'm able to run an rsync command that syncs one-way and I can extend that to sync 2-way:
wsl rsync -rlptzv --progress --exclude=.git '$PWD' 'marvink#s-dev01:~/code/new-project'
This needs to be ran locally but I can't find a way to do that. I'd need to run a task locally for example, but that isn't possibly when working on a remote (https://github.com/microsoft/vscode-remote-release/issues/168).
The other way around doesn't seem like an option to me as I don't want to expose any ports on my laptop and firewalls would get in the way depending on where I am.
My question
My workflow still seems a bit convoluted so I'm open to suggestions on that end but any ideas on how to sync my workspace files?
You don't need a local version of your code (containing the .devcontainer folder) if you're storing that code on the remote server. You should be able to open an ssh target in VScode using the Remote - SSH extension, which is the recommended approach in the link you added. The extension Remote - Containers 'stacks' on top of the SSH extension, so once connected over SSH you then connect to the container using the .devcontainer.json configuration located on your remote server.
If you don't want to use the extension and use a bind mount + specify docker.host in your settings.json file, you can sync code using the approaches in that same article, through SSHFS, docker machine, or rsync.
I have Linux Droplet on Digital Ocean, and I want to run on it some services – like SpringbootWeb and React.js.
Clearly I need to run the servers all the time, without being depends
on the terminal on/off (I’m using Putty) and I am planning to do it ,
by using nohup.
I saw other methods like those
In spring boot (See 3. Installing Spring Boot Applications) and in npm.
But I prefer for now use nohup since it’s easier and simpler.
I there is problem with that approach and it considered “bad practice” for production ?
(And if does, what considered a good practice ?)
Edit
Now seeing that nohup not saving react running after closing Putty-console
found also this idea for deploying React on nginx. (Digital Ocean run nginx)
There's nothing wrong with it, but you would still need to create some sort of init script to start your app on boot and stop it on shutdown.
So on a Linux system you would typically want to use systemd unit files for this, and have the init system handle the lifecycle of your server application. The reference guide mentions it here, or refer to this as a more complete example.
The task: Run a tensorflow train.py script I wrote in the cloud with at least 32GB of memory.
Requirements: The script has some dependencies like numpy, scipy, and mkt. I need to be able to install these. I just want a no-nonsense ssh shell like experience. I want to put all my files including the training data in a directory, pip install the packages if necessary, then just hit python train.py and let it run. I'm not looking to run a web app or have Google's machine learning platform do it for me.
All the tutorials around seem needlessly complicated, like they're meant for scaled deployments with http requests and all that. I'm looking for a simple way to run code on a server since my computer is too weak for machine learning.
Don't use AppEngine -- use Compute Engine instead. Almost the same thing, but very simple and you are completely in control of what you run, what you install etc.
Simple steps that should work for you:
-Create a Compute Engine instance
-Chose operating system (Ubuntu xx, but you can choose others instead)
-Chose how many CPUs and how much memory you want (select Customize in order to set yourself the CPU/memory ratio rather than getting default options)
-Enable HTTP/HTTPs in order to be able to use Tensorboard later
-Once created, SSH into the machine. Python is already pre-installed (2.7 default, but 3.x also available as Python3 alias)
-Install Tensorflow, Numpy, Pandas, and whatever you want with simple PIP
-You can also install Bazel if you want to build Tensorflow from source and to speed up the CPU operations
-Install gcsfuse if you want to copy/paste stuff quickly from cloud storage buckets
-Use tmux if you want to run several Tensorflow sessions in parallel (i.e.to try different hyperparameters/etc.)
This is all very clean and simple and works really well. Don't forget to shut it down after finished. You can also create a Preemptable instance to make it super-cheap (but it can be shut down at any time without warning, but happens rarely).
If it's possible, I'm interested in being able to embed a PostgreSQL database, similar to sqllite. I've read that it's not possible. I'm no database expert though, so I want to hear from you.
Essentially I want PostgreSQL without all the configuration and installation. If it's possible, tell me how.
Run postgresql in a background process.
Start a separate thread in your application that would start a postgresql server in local mode either by binding it to localhost with some random free port or by using sockets (does windows support sockets?). That should be fairly easy, something like:
system("C:\Program Files\MyApplication\pgsql\postgres.exe -D C:\Documents and Settings\User\Local Settings\MyApplication\database -h 127.0.0.1 -p 12345");
and then just connect to 127.0.0.1:12345.
When your application quits, you can always send a SIGTERM to your thread and then wait a few seconds for postgresql to quit (ie join the thread).
PS: You can also use pg_ctl to control your "embedded" database, even without threads, just do a "pg_ctl start" (with appropriate options) when starting the application and "pg_ctl stop" when quitting it.
You cannot embed it, nor should you try.
For embedding you should use sqlite as you mentioned or firebird rdbms.
Unless you do a major rewrite of code, it is not possible to run Postgres "embedded". Either run it as a separate process or use something else. SQLite is an excellent choice. But there are others. MySQL has an embedded version. See it at http://mysql.com/oem/. Also several java choices, and Mac has Core Data you can write too. Hell, you can even use FoxPro. What OS you on and what services you need from the database?
You can't embed it as a in process type thing like sqlite etc, but you can easily embed it into your application setup using Inno setup at http://www.innosetup.org. Search their mailing list archive and you will find someone did most of the work for you and all you have to to is grab the zipped distro and you can easily have postgresql installed when the user installs your app. You can then use the pg_hba.conf file to restrict the server to local host only. Not a true embedded DB, but it would work.
PostgreSQL is intended to run as a stand-alone server; it's probably possible to embed it if you hack at it hard and long enough, but it would be much easier to just run it as intended in a separate process.
HSQLDB (http://hsqldb.org/) is another db which is easily embedded. Requires Java, but is an excellent and often-used choice for Java applications.
Anyone tried on Mac OS X:
http://pagesperso-orange.fr/bruno.gaufier/xhtml/prod_postgresql.xhtml
http://www.macosxguru.net/article.php?story=20041119135924825
(Of course sqlite would be my embedded db of choice as well)
Well, I know this is a very very very old post, but if anyone has nowadays this question, I would refer to:
You can use containers running Postgres. Here's a post that could be helpful, doing something along this line using R:
https://rsangole.netlify.app/post/2021/08/07/docker-based-rstudio-postgres/?utm_source=pocket_mylist
Take a look at duckdb https://duckdb.org/docs/installation/ It is relatively new and still needs to mature. But it works pretty much like an embedded database ("In-process, serverless"), with bindings for several languages (Python, R, Java, ...)