How to setup Flink Local Cluster - apache-flink

I am trying to use Flink local on Linux and Windows, for my bachelor
thesis. I have found these steps for local setup:
https://ci.apache.org/projects/flink/flink-docs-release-1.1/quickstart/setup_quickstart.html#start-a-local-flink-cluster
When I try this I got only errors like this:
-bash: bin/start-local.sh: No such file or directory
When I go to the directory of the start-local.sh file then I got
/flink-1.1.2/flink-dist/src/main/flink-bin/conf/flink-conf.yaml: No such file or directory
Same problem with Windows.
What do I have to change so that it works?

It seems that you have downloaded the sources. It is necessary to download one of binaries from here: https://flink.apache.org/downloads.html#binaries. Then, follow the given instructions for local setup.
Of course if you want to build Flink from sources, use this guide: https://github.com/apache/flink#building-apache-flink-from-source.

Related

net/rime.h error using Cooja, how can I install it but not just download it?

I am trying to do a simple simulation in Cooja (literally the example on the Contiki website here).
However I have an error message linked to my #include "net/rime.h", Cooja outputs the following message while compiling:
fatal error: net/rime.h: No such file or directory
I thought that it came with Contiki when I've installed contiki-ng following the procedure on the official website. My quick fix was to download the file from the github repository and store it in my active directory. However I find it quite ugly, is there a way to install this library ? Because I haven't found a way yet.
Thanks a lot.
It's a question close to this one.
Which kind of method are you using to start Cooja? Directly on the SO? Over Docker?
I recommend you de Docker way, and remember to apply the command
git submodule update --init --recursive
after cloning the git repository.
Also, if you are using Docker, check if the binding of the Contiki folders between de Host and the Container are correctly made.

Where can I download mobilenet_v1_160res_0.5_imagenet_labels.txt for local testing?

For our course projects, we started off with Google's AIY Vision kit. I am looking into how to use output for next level processing. We are all newbies to ML
I am testing image classification using sources from Github AIY Projects on my mac.
python image_classification.py -i images/cat01.jpeg
FileNotFoundError: [Errno 2] No such file or directory: '/opt/aiy/models/mobilenet_v1_160res_0.5_imagenet_labels.txt'
I would like to test this to see output format that I want to use for next steps.
Where can I get the labels txt file?
I found a good Coral link that has TPU models and labels. Is there a way to use this instead?
couldn't get it to work on my mac but tried to create a dockerfile for some of the AIY example codes to run on a docker host within the mac.
The missing piece to overcome the error you listed above for me was to :
apt-get install -y aiy-models after adding the Debian repo source. Not sure how to get around that on the mac.

KNIME Command Line Execution - ClassNotFoundException

I'd like to schedule a KNIME workflow. The workflow does its job very good as long as I start it from the KNIME GUI application. When I execute the same workflow via command line, java complains that com.microsoft.sqlserver.jdbc.SQLServerDriver
could not be found (ClassNotFoundException).
I invoke it via:
"D:\Progamme\KNIME\knime.exe" -nosplash -application -consoleLog org.knime.product.KNIME_BATCH_APPLICATION -preferences="absolutepathto\preferences.epf" -workflowDir="absolutepathto\workflow"
Since the error message signals missing content in the java CLASSPATH I also tried to add the parameters
-vmargs -classpath .;"absolutepathto/sqljdbc42.jar"
But still I earn a java slap, pointing to the same error...
I also tried to run the command from within the knime.exe's directory and I also tried to add the JAR file to Preferences -> Java -> Build Path -> Classpath Variable / User Libraries (referenced via the -preference argument). But that had no effect.
Did anybody face the same problems? Maybe with other third party JARs?
It is all about a Database connector that is configured like this:
Does the integrated security maybe force a misleading error?
System spec: KNIME 3.2.2 on Windows Server 2008 R2
Update - extract from preferences file
/configuration/org.eclipse.core.net/org.eclipse.core.net.hasMigrated=true
/configuration/org.eclipse.ui.ide/MAX_RECENT_WORKSPACES=10
/configuration/org.eclipse.ui.ide/RECENT_WORKSPACES=<list of some workspaces>
/configuration/org.eclipse.ui.ide/RECENT_WORKSPACES_PROTOCOL=3
/configuration/org.eclipse.ui.ide/SHOW_RECENT_WORKSPACES=false
/configuration/org.eclipse.ui.ide/SHOW_WORKSPACE_SELECTION_DIALOG=true
Is there maybe a problem due to the fact that it is a shared KNIME instance among several users and the command line execution does not know which workspace has to be chosen? Is the workspace somehow needed and why?
Partial Solution:
I finally managed it but I don't know exactly why it works now. What I did was to load a fresh portable version of KNIME and ran the same commands only changing the executable path to the new portable version. Before that I started the portable version once to set the workspace directory and register the database driver in preferences dialog and .ini file, nothing else, same configuration so far as the shared KNIME instance. What I am really wondering abpout is that from now on the commands are also working with the shared KNIME instance. I really don't know what caused the change that let KNIME find the driver class.
Info
Because I encountered a few more problems within shared environment in KNIME command line mode, that led to undeterministic execution results, I wrote a little .NET library. This gives me more flexibility/control over the workflow execution (which returncodes and error messages occured and so on). You can find it here if you're interested: KnimeNet
I took a very minimal approach:
cd "C:\Program Files\KNIME"
.\knime -nosplash -noexit -consoleLog -reset -application org.knime.product.KNIME_BATCH_APPLICATION -workflowFile="D:\Work\Knime Workflows\Output\CMD_Test.knwf" -preferences="D:\Work\Knime Workflows\Output\CMD_Test.epf"

java eclipse hadoop map reduce program unable to access my files stored in hdfs

My java eclipse hadoop map reduce program is displaying an error unable to locate the input file. I had copied the files to hadoop directory via terminal using hadoop commands. I can see the files in java eclipse dfs location. And also using the command hadoop dfs -ls in terminal. When i created a normal folder (not hdfs) then the problem get solved. But then program is accesing the file from local file system.
I had installed hadoop 1.2.1 on redhat server 32 bit, using java eclipse luna, i had already included hadoop plugins and external jar files from the hadoop library. Input and output path are given through run time arguments
First of all,Hadoop eclipse plugins doesn't have great reliability. I had the same problem when using the plugin with the Eclipse Luna. But that compatibility issue got solved when i used Eclipse Juno. And there is no suitable plugin available for Hadoop 2.x versions.
You can use the tool Maven to manage all the Hadoop dependencies just like Hadoop eclipse plugin except that you should run the Job from the terminal.
Link on how to use Maven with Hadoop
Accept my answer if it fits your case. :)

How extract the package source into the some other directory in buildroot

Here my package is net-snmp.
Here is the task : Addition of Net-SNMP Source directory at path personal/apps/snmp/
Requirement is that it should download from website if there is change in version name and it should patch , configure and build soruce.
Hence, for that we need to configure Config.in and netsnmp.mk files in build/package/netsnmp in order to fulfill this requirement.
By adding following configuration in netsnmp.mk
NETSNMP_VERSION = 5.7.2.1
NETSNMP_SOURCE = net-snmp-$(NETSNMP_VERSION).tar.gz
NETSNMP_SITE = http://www.sourceforge.net/projects/net-snmp/files/net-snmp/$(NETSNMP_VERSION)
We can download the latest net-snmp tar ball which will be downloaded in dl/ folder in buildroot.
From here it will untar in build/output/build and will patch configure and build. After that it will build the executable at appropriate location inside target folder.
Here the issue is I want netsnmp source code to be at personal/apps/snmp folder which seems to be problematic and can edit the source in path personal/apps/snmp/ and configure build from here.
Any help and suggestion will be appreciated.
There is already a netsmp package in Buildroot. Why do you want to create another one?
Also, the Buildroot community is going to be much more reactive if you ask questions on the project's mailing list.

Resources