I have a Docker container running a C application using hiredis, which should write data to a Redis server exposed at the default address and port running locally on the same Linux device at 127.0.0.1:6379 .
The Redis server is running in a different Docker container. I start this container running, exposing port 6379 as follows : sudo docker run --name redis_container -d -p 6379:6379 40c68ed3a4d2
redsi-cli can connect to this via 127.0.0.1:6379 without issues.
However, no matter what I try, my container which should write to the Redis gets a Redis connection refused error from the C code all the time. This was my last attempt at running the container : sudo docker run --expose=6379 -i 7340dfee8ea5
What exactly am I missing here? Thanks
The C client is running inside a container, that means 127.0.0.1 points to the container itself, not to your host. You should configure the redis client to redis_container:6379 as that is the name you have used when docker run the redis container. More about this here
Besides, both containers need to be inside the same docker network. Use the following command to create a simple network
docker network create my-net
and add --network my-net to both docker run commands (redis client and redis server)
You can read more about docker network here
I am using an Azure DSVM in a DevTest Lab running Windows Server 2019. I am trying to get Docker installed and working to allow me to run local experiments from Azure ML Service environments.
I want to build a custom Linux container on Docker - which I believe is possible on Windows from reading some other online posts (I can't use a Linux host for various reasons). When I try to create such an image that contains a WORKDIR ... step, I get a "container ***** encountered an error during CreateProcess: failure in a Windows system call" error.
I installed Docker on the DSVM (which is a Standard D2s_v3) by adding the "Docker" artifact at creation and then running the following commands to enable Linux containers:
$> Install-WindowsFeature -Name Hyper-V -IncludeManagementTools -Restart
$> [Environment]::SetEnvironmentVariable("LCOW_SUPPORTED", "1", "Machine")
Running a simple Linux container works fine:
$> docker run --rm -it alpine:latest
/ # ls
bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var
/ #
To build a custom image, I'm using a simple Dockerfile as follows:
FROM alpine:latest
WORKDIR /abm
The image appears to build successfully:
$> docker build --no-cache -t abm-alpine:workdir -f .\abm-alpine.Dockerfile .
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM alpine:latest
---> a187dde48cd2
Step 2/2 : WORKDIR /abm
---> 495f8ecb3a0e
Removing intermediate container 219e91296e47
Successfully built 495f8ecb3a0e
Successfully tagged abm-alpine:workdir
When I run the image, I get the following error:
$> docker run --rm -it abm-alpine:workdir
C:\Program Files\Docker\docker.exe: Error response from daemon: container 01fad57c971d672d91238a6c6ec21376e033006ec4c26563e91e7288cfb3bfeb encountered an error during CreateProcess: failure in a Windows system call: The virtual machine or container exited unexpectedly. (0xc0370106) extra info: {"CommandArgs":["/bin/sh"],"WorkingDirectory":"/abm","Environment":{"HOSTNAME":"01fad57c971d","PATH":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","TERM":"xterm"},"EmulateConsole":true,"CreateStdInPipe":true,"CreateStdOutPipe":true,"ConsoleSize":[50,120],"OCISpecification":{"ociVersion":"1.0.0","process":{"terminal":true,"consoleSize":{"height":50,"width":120},"user":{"uid":0,"gid":0},"args":["/bin/sh"],"env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME=01fad57c971d","TERM=xterm"],"cwd":"/abm","capabilities":{"bounding":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"effective":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"inheritable":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"permitted":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"]}},"root":{"path":"rootfs"},"hostname":"01fad57c971d","mounts":[{"destination":"/proc","type":"proc","source":"proc","options":["nosuid","noexec","nodev"]},{"destination":"/dev","type":"tmpfs","source":"tmpfs","options":["nosuid","strictatime","mode=755","size=65536k"]},{"destination":"/dev/pts","type":"devpts","source":"devpts","options":["nosuid","noexec","newinstance","ptmxmode=0666","mode=0620","gid=5"]},{"destination":"/sys","type":"sysfs","source":"sysfs","options":["nosuid","noexec","nodev","ro"]},{"destination":"/sys/fs/cgroup","type":"cgroup","source":"cgroup","options":["ro","nosuid","noexec","nodev"]},{"destination":"/dev/mqueue","type":"mqueue","source":"mqueue","options":["nosuid","noexec","nodev"]},{"destination":"/dev/shm","type":"tmpfs","source":"shm","options":["nosuid","noexec","nodev","mode=1777"]}],"linux":{"resources":{"devices":[{"allow":false,"access":"rwm"},{"allow":true,"type":"c","major":1,"minor":5,"access":"rwm"},{"allow":true,"type":"c","major":1,"minor":3,"access":"rwm"},{"allow":true,"type":"c","major":1,"minor":9,"access":"rwm"},{"allow":true,"type":"c","major":1,"minor":8,"access":"rwm"},{"allow":true,"type":"c","major":5,"minor":0,"access":"rwm"},{"allow":true,"type":"c","major":5,"minor":1,"access":"rwm"},{"allow":false,"type":"c","major":10,"minor":229,"access":"rwm"}]},"namespaces":[{"type":"mount"},{"type":"network"},{"type":"uts"},{"type":"pid"},{"type":"ipc"}],"maskedPaths":["/proc/kcore","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug"],"readonlyPaths":["/proc/asound","/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]},"windows":{"layerFolders":["C:\\ProgramData\\docker\\lcow\\5ba6a7b4fbdf9748ec89898be9bdaa911ee614436a475945638ab296b1155966","C:\\ProgramData\\docker\\lcow\\01fad57c971d672d91238a6c6ec21376e033006ec4c26563e91e7288cfb3bfeb"],"hyperv":{},"network":{"endpointList":["D615E3D5-B6AA-401E-A0A0-72581FA47059"],"allowUnqualifiedDNSQuery":true}}}}.
I've tried various logs (e.g. Get-WinEvent -LogName Microsoft-Windows-Hyper-V-Compute-Operational and Get-EventLog -LogName Application -Source Docker) but cannot see any additional information about the error.
Can anyone advise if it is possible to create custom Linux-based images on a Windows DSVM? If it is, can anyone advise what the problem may be or any additional troubleshooting steps I could take?
Thanks!
It is possible to create Linux container on Windows Server.
Although this is currently in experimental stage.
This article might help : https://www.b2-4ac.com/lcow-linux-containers-on-windows-server/
I've created a docker container that contains a mssql Database. On the command line ip a gives an ip address for the container, however trying to ssh into it username#docker_ip_address yields ssh: connect to host ip_address port 22: Connection refused. So I'm wondering if I am even able to ssh into the container so I don't have to always be using the docker tool docker exec .... and if so how would I go about doing that?
To ssh into container you should full-fill followings
SSH server(Openssh) should be installed within the container and ssh service should be running
Port 22 should be published from container (when you run the container).more info here > Publish ports on Docker
docker ps command should display mapped ports 22
Hope above information helps for you to understand the situation...
If your container contains a database server, the normal way to interact with will be through an SQL client that connects to it; Google suggests SQL Server Management Studio and that connector libraries exist for popular languages. I'm not clear what you would do given a shell in the container, and my main recommendation here would be to focus on working with the server in the normal way.
Docker containers normally run a single process, and that's normally the main server process. In this case, the container runs only SQL Server. As some other answers here suggest, you'd need to significantly rearchitect the container to even have it be possible to run an ssh daemon, at which point you need to worry about a bunch of other things like ssh host keys and user accounts and passwords that a typical Docker image doesn't think about at all.
Also note that the Docker-internal IP address (what you got from ip addr; what docker inspect might tell you) is essentially useless. There are always better ways to reach a container (using inter-container DNS to communicate between containers; using the host's IP address or DNS name to reach published ports from the same or other hosts).
Basically, alter your Dockerfile to something like the following - that will install openssh-server, alter a prohibitive default configs and start the service:
# FROM a-image-with-mssql
RUN echo "root:toor" | chpasswd
RUN apt-get update
RUN apt-get install -y openssh-server
COPY entrypoint.sh .
RUN cd /;wget https://gist.githubusercontent.com/spekulant/e04521d6c6e1ccffbd3455c673518c5b/raw/1e4f6f2cb32caf3a4a9f73b02efdcbd5dde4ba7a/sshd_config
RUN rm /etc/ssh/sshd_config; cp sshd_config /etc/ssh/
ENTRYPOINT ["./entrypoint.sh"]
# further commands
Now you've got yourself an image with ssh server inside, all you have to do is start the service, you cant do RUN service ssh start because it won't work - docker specifics, refer to the documentation. You have to use a Entrypoint like the following:
#!/bin/bash
set -e
sh -c 'service ssh start'
exec "$#"
Put it in a file entrypoint.sh next to your Dockerfile - remember to chmod 755 entrypoint.sh it. There's one thing to mention here, you still wouldn't be able to ssh into the container - the default SSH server configuration doesn't allow login into root account using a password. So you either change the configs yourself and provide it to the image, or you can trust me and use the file I created - inspect it with the link from Dockerfile - nothing malicious there, only a change from prohibit-password to yes.
Fortunately for us - MSSQL official images start from Ubuntu so all the commands above fit perfectly into the environment.
Edit
Be sure to ask if something is unclear or I'm jumping too fast.
I'm new to nifi and i want to connect SQL server database to nifi and create a data flow with the processors. how can I do this, can any one Help me with this clearly.
Thanks in Advance
sam
Here are two great articles on getting information in and out of databases with NiFi:
http://www.batchiq.com/database-injest-with-nifi.html
http://www.batchiq.com/database-extract-with-nifi.html
They describe/illustrate how to configure a DBCPConnectionPool service to provide connection(s) to an RDBMS, and example flows to extract data and ingest data.
Expanding on mattyb answer
If you are using the latest Hortonworks sandbox, or other setup that uses docker containers, read below.
You have to install the JDBC jar file inside the docker. For SQL Server, it should be 6.2 or above.
docker ps
docker exec -it <mycontainer uuid> bash
How do I get into a Docker container's shell?
will help you log into the container.
cd file:///usr/lib/jvm/jre/lib/
mkdir jdbc
cd ./jdbc
wget https://download.microsoft.com/download/3/F/7/3F74A9B9-C5F0-43EA-A721-07DA590FD186/sqljdbc_6.2.2.0_enu.tar.gz
tar xvzf sqljdbc_6.2.2.0_enu.tar.gz
cp ./sqljdbc_6.2/enu/mssql-jdbc-6.2.2.jre8.jar ./
jdbc:sqlserver://192.168.1.201:1433;databaseName=[your database]
com.microsoft.sqlserver.jdbc.SQLServerDriver
You might need to replace the ip address with IPv4 address of your host found under ipconfig in Windows or ifconfig in Mac/Linux.
You may change file:///usr/lib/jvm/jre/lib/ to any path you desire.
Expanding on TamusJRoyce's answer
If you are running nifi via a docker image like apache/nifi or the aforementioned Hortonworks sandbox, the following should help you get the required driver on the image so that you don't need to exec into the container to do it manually.
See the comments below the docker file
FROM apache/nifi
USER root
RUN mkdir /lib/jdbc
WORKDIR /lib/jdbc
RUN wget https://download.microsoft.com/download/3/F/7/3F74A9B9-C5F0-43EA-A721-07DA590FD186/sqljdbc_6.2.2.0_enu.tar.gz
RUN tar xvzf sqljdbc_6.2.2.0_enu.tar.gz
RUN cp ./sqljdbc_6.2/enu/mssql-jdbc-6.2.2.jre8.jar ./
USER nifi
EXPOSE 8080 8443 10000 8000
WORKDIR ${NIFI_HOME}
ENTRYPOINT ["../scripts/start.sh"]
The above image uses apache/nifi as the base image. You can use any nifi docker image has a base if you would like.
You can specify any location for lib/jdbc, just remember that you need to use this as the reference for the file location so that it is referenced as file:///lib/jdbc/mssql-jdbc-6.2.2.jre8.jar
Lastly, switch back to the nifi user and finish off with the standard nifi image details. This will allow the image to run correctly.