We use pytorch in a nvidia/cuda:xx.x-base-ubuntu18.04 (base) image with a final size of ~6 GB.
I know that nvcc is available in nvidia/cuda:xx.x-devel (devel) tag, but it increases the image size to > 10 GB.
Can I install nvcc only in the base image?
nividia provided installers have command line options.
this will allow to install selected components. (such as with or without graphics drivers, samples, docs, ...)
Related
When I try to add it to sources as per debian install instructions I get this error. I'm guessing this means that there are no arm packages for it.
Failed to fetch https://dist.crystal-lang.org/apt/dists/crystal/InRelease Unable to find expected entry 'main/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file)
I'm guessing I probably need to install it from source. How would I go about doing that with an arm cpu? When I check it out and run make I get the error:
You need to have a crystal executable in your path! Makefile:113:
recipe for target '.build/crystal' failed make: *** [.build/crystal]
Error 1
Any suggestions would be greatly appreciated.
EDIT: There's now a semi-official repository for crystal on raspbian, check it out here: http://public.portalier.com/raspbian
Crystal doesn't build Debian packages for ARM, and you're correct in that you'll need to build from source.
However, the Crystal compiler is written in Crystal. This presents the obvious problem of how to get a compiler to build the compiler. The answer is cross-compilation: building an arm binary on a x86 desktop computer and copying it across.
Here's a quick step-by-step based on my memory of last time I cross-compiled:
Install Crystal on a x86 desktop PC, and check it works.
Install all required libraries on the desktop and Raspberry Pi. You'll need the same LLVM version on the Raspberry Pi and desktop. This is probably the hardest and longest step. You can install llvm 3.9 from debian testing for ARM, see this stackoverflow post for how to install only LLVM from debian testing.
Check out the sources from git on both computers, and run make deps.
Cross-compile the compiler by running this command in the root of the git repository:
./bin/crystal build src/compiler/crystal.cr --cross-compile --target arm-unknown-linux-gnueabihf --release -s -D without_openssl -D without_zlib
This command will create a crystal.o file in your current directory, and also output a linker command (cc crystal.o -o crystal ...).
Copy crystal.o to the raspberry pi, and run the linker command. Be sure to edit the absolute path to llvm_ext.o so that it points to the Crystal checkout on your Raspberry Pi, not the checkout on your desktop. Also make sure that all references to llvm-config in the command are for the correct LLVM version. For example, changing /usr/local/bin/llvm-config to llvm-config-3.9 on Raspbian.
Run the crystal executable in your current directory (./crystal -v) and make sure it works.
Ensure to set CRYSTAL_PATH environment variable is set to lib:/path/to/crystal/source/checkout/src so that the compiler can find the standard library when compiling applications.
I am using ubuntu, but the question is for linux in general.
I installed a module/driver by compiling my linux kernel and install the new compiled kernel. It works fine.
In order to make this driver work in another machine without installing the new kernel, I copy the .ko file to the new machine under /lib/modules/<version>/... and then run sudo depmod -a. Then run sudo modprobe <drivername>. The module can be loaded without a problem. but the device is not working well with this .ko module.
The two machines are not identical to hardwares, BUT they are identical to kernel version and ubuntu release version. Normally, copying .ko file should work for the same linux release and the same kernel.
More information about the driver. it's a hid pen tablet driver. All patch files:
one .c file in drivers/hid/
add one line in drivers/hid/Makefile
add a few lines to drivers/hid/usbhid/Kconfig
add a few lines to drivers/hid/hid-ids.h
add a few lines to drivers/hid/usbhid/hid-quirks.c's hid_blacklist struct before { 0, 0 }
That's all.
I even tried to copy the entire drivers/hid/ directory includig all the .ko files from the first machine to the second one. but no luck. The pen tablet can be recognized in the second machine, I am able to do mouse left click event with the pen, but the pen can not move the cursor.
Hopefully, I provided enough details. My goal is to only install the module to identical linux release (kernel) without reinstalling the kernel. I am not sure how to achieve that or if it's possible.
Thanks a lot.
PS:
The dmesg output in 1st machine which works: http://paste.ubuntu.com/6419301/
The dmesg output in 2nd machine: http://paste.ubuntu.com/6419302/
In 1st machine, before plugging in the tablet, lsmod doesn't show the module. after plugging in, the module can be loaded automatically. I can see lsmod shows the module.
In 2nd mahcine, the module can not be loaded automatically by plugging in the device. I have to do sudo modprobe <module> manually.
Since I will have to install the module to many machines in my company, it's easier to install the module without reinstalling the kernel. I tried to install the kernel .deb packages which built in the 1st machine to the 2nd machine, it works fine in 2nd machine. but I don't feel good to reinstall the kernel to many machines. Thanks.
It seems the kernel you built isn't a 1:1 match. Also, generally there's no need to compile a new kernel.
The simplest way to deal with an out-of-tree driver deployment is to use DKMS.
What you need to provide is just a dkms.conf file specifying the package name, version, and driver names and destinations (within /lib/modules/{kernel}).
In the following examples, things within braces need to be replaced with the real thing, e.g. if version is 1.0.0, then {version} with 1.0.0, obviously.
Example dkms.conf:
PACKAGE_NAME="{mydriver}"
PACKAGE_VERSION="{version}"
BUILT_MODULE_NAME[0]="{mydriver}"
BUILT_MODULE_LOCATION[0]="/{mycompany?}"
AUTOINSTALL="yes"
Then you just need to install the sources to /usr/src/{mydriver}-{version}, and run dkms:
dkms add -m {mydriver} -v {version}
dkms build -m {mydriver} -v {version}
dkms install -m {mydriver} -v {version}
You should take a look at what other people have done in this area, there's a great deal of automation you can apply to testing and release processes. Bluecherry's solo6x10 out-of-tree version provides some useful make targets (disclosure: I'm the one who wrote that).
Also, you definitely want to build and distribute packages, you can use solo6x10/debian as a template, and you can read about repositories in the Debian wiki.
You can add the module to /etc/modules so it's loaded at boot time.
I have downloaded and ran the CUDA 5.0 installer on my Mint 15 64bit distro. After hours of agony adjusting / removing / installing packages, it was able to finish installation - at least that what it said.
I can go run the CUDA samples so I thought hey it's working. However, I just made a new cu file and wanted to compile but it said "nvcc command not found"
I have looked at a topic similar to this here and they are talking about /opt/bin/ directory however on mine, there is no such directory. Does that mean it actually did not install ? It tells me to install nvidia cuda toolkit with apt-get but I am not sure if I should do that.
Also, I did say I ran the CUDA samples fine but I have to say ldconfig /usr/local/cuda/lib64
before I can get it to working. Is there a way to automate that ?
Thanks
You need to add the bin directory of the nvcc compiler driver to your PATH (environment variable), and you need to add the appropriate lib directories to your LD_LIBRARY_PATH environment variable.
For an immediate test, this should be as simple as:
export PATH=$PATH:/usr/local/cuda/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/lib
These instructions should be presented to you at the completion of a successful cuda toolkit install, but it seems your install method may have been roundabout.
To make this "automatic" you may want to investigate one of the methods to add these statements to a script run at login. For example, if you have a .bashrc file in your user's home directory, try editing that with the above commands. It should probably be sufficient to put the above commands at the very end of your ~/.bashrc file if you have one.
Note that Linux Mint is not one of the officially supported CUDA distros, so your mileage may vary.
I'm doing a series of benchmarks and found the httpperf tool.
But the version in my ubuntu 12.04 has a too small file descriptor size. Because it warns me with this message:
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
There used to be guide to compile httperf with a bigger size in http://gom-jabbar.org/articles/2009/02/04/httperf-and-file-descriptors but the site is down now.
Does anyone knows the steps to compile the tool with the proper settings?
I've always followed the instructions here, which should set the global values properly. You can check by issuing a ulimit -n (N.B. I had to include ulimit -n 65535 in my .profile — for some reason named users don't require this but root does.)
Don't forget to recompile httperf. Before doing make install issue a ./httperf -v | grep maximum — you should see 65535. If not, something went wrong.
I am working on a similar project (httperf 0.9.0 on Ubuntu 12.04) but I am having some difficulty getting httperf to actually compile properly. I'm sure I've forgotten something basic, but let me know how you fare. EDIT: Realized my problem was a library version incompatibility. I imported the binary built on a different server and it works fine.
I am using Cygwin with gcc 4.5 to compile a C program. I am using Windows XP. The size of Cygwin package is too large i.e. about to 900MB, and I want a gcc compiler of less size.
Is there any gcc compiler of size max 10-15MB to compile C program on windows.
Or I can reduce the size of Cygwin package.
Actually there are lot of extra things present in Cygwin package but I don't know which files to remove or keep.
Thanks in advance.
If all you need in cygwin is the gcc package, run the setup.exe and then install select only the "Base" packages and "gcc4" and "gcc4-core" from the "Devel" set. Once you hit the "Next" button it may provide a list of required packages due to dependency. Install all of them and see if it meets your size requirement.