Why is Jena tdb2.tdbquery optimization stuck on "Reorder/generic" - query-optimization

I am using apache-jena-4.5.0 and fuseki pretty much out-of-the-box. I had created a TDB2 dataset using fuseki, but now shut it off and using command-line utilities of jena on a Windows box inside a bash shell.
My basic command is:
java -cp "*" tdb2.tdbquery --loc ~/path/to/databases/DEMO--explain --set arq:logExec=FINE --time --query ~/path/to/demoquery.txt
And my question is why does the output always contain only Reorder/generic like this:
15:56:00 INFO exec :: Reorder/generic
Even after I have tried all these:
successfully run tdb2.tdbstats and gotten a reasonable-looking temp.opt file as output
moved that temp.opt to each of /path/to/DEMO/stats.opt and /path/to/DEMO/Data-001/stats.opt
tried uppercase STATS.OPT for each since I'm on windows, just to be sure
Still I don't seem to be able to produce any output with Reorder/stats

This question did not contain enough detail to answer. The intended question was why won't TDB2 optimize my query and the answer was in the SPARQL, not in the invocation of tdb2.tdbquery or the location of the stats.opt file.
My SPARQL contained multiple FROM clauses, which forced TDB into BGP mode (instead of quads) and thwarted any optimization. As best we can tell at the moment, one wishing to use the TDB2 optimizer should use either the default graph, or a combination of FROM NAMED and GRAPH which causes the evaluation of graphs one at a time.

Related

How to check whether large command text answer contains a specific string

This is related to u-boot.
I looked at this link (How to test the return of a command in U-Boot CLI) but it doesn't cover what I need. I also checked some other pages related to the 'test' command in u-boot but I can't figure out how to do what I would like. So there it is.
I have a u-boot command on a SBC which returns a set of information as text printed on the screen, where some part are strings representing versions. What I would like is to check whether the command return contains some specific strings to choose automatically what to do.
I have seen a few 'test' command help, but I couldn't figure out how to do this type of check on the text returned by this command. My goal is to have a set of nested ifs to choose in the different cases, or anything equivalent. If regex can be used, they perhaps could be helpful.
Does anybody know how to do this?
Thank you.
In U-Boot's hush shell there is no support for pipes. There is also no command like grep. So there are no means to succeed via U-Boot's built-in shell commands.
If you have access to the source of the command that you are executing, you could change it and use function env_set() to write the relevant data to environment variables and then use test for your purposes.

SPSS: Use index variable inside quotation marks

I have several datasets over which i want to run identical commands.
My basic idea is to create a vector with the names of the datasets and loop over it, using the specified name in my GET command:
VECTOR=(9) D = Name1 to Name9.
LOOP #i = 1 to 9.
GET
FILE = Directory\D(#i).sav
VALUE LABELS V1 to V8 'some text D(#i)'
LOOP END.
Now SPSS doesn't recognize that i want it to use the specific value of the vector D.
In Stata i'd use
local D(V1 to V8)
foreach D{
....`D' .....
}
You can't use VECTOR in this way i.e. using GET command within a VECTOR/LOOP loop.
However you can use DEFINE/!ENDDEFINE. This is SPSS's native macro facility language, if you are not aware of this, you'll most likely need to do a lot of reading on it and understand it's syntax usage.
Here's an example:
DEFINE !RunJob ()
!DO !i !IN 1 !TO 9
GET FILE = !CONCAT("Directory\D(",#i,").sav").
VALUE LABELS V1 to V8 !QUOTE(!ONCAT("some text D(",#i,")",
!DOEND
!ENDDEFINE.
SET MPRINT ON.
!RunJob.
SET MPRINT OFF.
All the code between DEFINE and !ENDDEFINE is the body of the macro and the syntax near to the end !RunJob. then runs and executes those procedures defined in the macro.
This a very simply use of a macro with no parameters/arguments assigned but there is scope for much more complexity.
If you are new to DEFINE/!ENDEFINE I would actually suggest you NOT invest time in learning this but instead learn Python Program ability which can be used to achieve the same (and much more) with relative ease compared to DEFINE/!ENDDEFINE.
A python solution to your example would look like this (you will need Python Programmability integration with your SPSS):
BEGIN PROGRAM.
for i in xrange(1,9+1):
spss.Submit("""
GET FILE = Directory\D(%(i)s).sav
VALUE LABELS V1 to V8 'some text D(%(i)s)'.""" % locals())
END PROGRAM.
As you will notice there is much more simplicity to the python solution.
#Caspar: use Python for SPSS for such jobs. SPSS macros have been long deprecated and had better be avoided.
If you use Python for this, you don't even have to type in the file names: you can simply look up all file names in some folder that end with ".sav" as shown in this example.
HTH!
The Python approach is as Ruben says much superior to the old macro facility, but you can use the SPSSINC PROCESS FILES extension command to do tasks like this without any need to know Python. PROCESS FILES is included in the Python Essentials in recent versions of Statistics but can be downloaded from the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in older versions.
The idea is that you create a syntax file that works on one data file, and PROCESS FILES iterates that over a list of input files or a wildcard specification. For each file, it defines file handles and macros that you can use in the syntax file to open and process the data.

Control output from makefile

I'm trying to write a makefile to replace one of the scripts used in building a fairly large application.
The current script compiles one file at a time, and the primary reason for using make is to parallelise the build process. Using make -j 16 I currently get a factor of 4 speedup on our office server.
But what I've lost is some readability of the output. The compilation program for a file bundles up a few bits and pieces of work, including running custom pre-compilers, and running the gcc command. Each of these steps outputs some information, and I would prefer it to buffer the output from the command, and then show the whole lot in one go.
Is it possible to make make do this?
If you upgrade to GNU make 4.0, then you can use the built-in output synchronization feature to get what you want.
If you don't want to upgrade, then you'll have to modify each of your recipes to be wrapped with a small program that manages the output. Or you can set the SHELL variable to something that does it for you. Searching the internet should give you some examples.
A simple way to accomplish this is to send all the log output a to log directory with each file named, say:
log_file_20131104_12013478_b.txt // log_file_<date>_<time>_<sequence letter>.txt
and then simply cat them all together as your last make job in the dependency chain:
cat log_dir/log_file_20131104_12013478_*.txt > log_file_20131104_12013478.txt
With makepp this is the default behaviour as soon as you use -j. All the individual outputs (and entering dir messages) get collected and are output together as soon as the command terminates.

Get full path of executable of running process on AIX

This is most similar to Get full path of executable of running process on HPUX…, except for AIX.
The basic question is: how, on AIX, can I determine the full path to the current executable? Having to do this before doing anything else (e.g., chdir) is fine.
The most accurate answer I've found so far is to check the output from
svmon -P $$ -O format=nolimit,filename=on,filtertype=client
(where $$ has its shell meaning: current pid). That's not only heavy amounts of C, but svmon is also not very fast and can easily overwhelm the runtime of the rest of the application.
The next best answer seems to be to simply look at argv[0], and, if it has a slash in it, it's either a full path name (starts with a leading /) or a relative-to-current-dir name (does not start with a leading /). If it doesn't have a slash in it, it's relative to something in PATH.
And if, after this resolution, I end up with a symlink, then there's all the resolution of symlink(s) to deal with as well (hard links are probably beyond the scope of any solution). This solution looks like it's relatively cross-platform, but is also very heavy in the C code (should be faster than svmon). And I expect there are race-conditions and such to deal with.
Thanks,
Update: I'm looking for a solution to submit to the perl devs. And they're going to worry about an insecure PATH, especially in setuid/setgid scenarios. See perlsec. I think that we could be okay here, but if you combined setuid with faking argv[0], you could force perl to think it's somewhere else, and load the wrong modules. The "next best" answer above only really works when not in perl's taint-mode.
Why can't you use ps options as a base line? Granted, you'll still need to process the cmd value to see if has a leading '/' or not. Something like
ps -o pid,env,cwd,cmd | grep youAppName | awk -f programToRationalizePathName
I don't have access to AIX anymore, but I did work on it for 2 1/2 years and I know I've used this sort of feature. I didn't think it was slow, but I might have had different requirements than you do.
I get the impression you want a utility function, a 1-at-time call that returns the full path, but if you need an on-going process and are concerned about re-starting ps every 1 minute (for example), look at the AIX specific nmon utility. Not sure if it can generate output similar to the ps -o cmd but it can be set up to run as long as you want, as often as you want (down to 1 second intervals) and it is just one process, whose output can be redirected as needed. (Nmon is not part of the std install, but most organizations do install it, as it is IBM blessed (if not supported directly)).
Of course all of the 'not 100%' caveats apply from the similar questions mentioned by you and in the comments.
I hope this helps.
Use ps to get executable path address
ps -aef | grep app | awk '{print $8}'
above command gives your app executable path address

Recording command line input and output on linux with C

Basically I want to do a program almost like a keylogger. The thing is that I as network admin sometimes I don't remember what I did to a machine on certain case, or same times I make howto's and tutorials for linux. I want to record what have i done.
So basically the idea of this program is:
you type the name of the program, (I call it rat for the moment)
$ rat
Welcome everything from now on will be recorded
recording $ ls
file1 file2 file3
recording $ quit
Bye bye
Everything you do will go out to an xml file. Something like this
<?xml version='1.0' encoding='UTF-8' ?>
<rat>
<command>
<input>ls</input>
<output>file1 file2 file3</output>
<err><err>
</command>
</rat>
i am doing some tests with fp_in = popen( input, "w");
and system, but first with popen i cant change directories and with "system i cant properly manage the input and output.
I was also checking if there is something I can do to bash like a plugin but haven't find any information.
At some points if feels like it I should create another shell (which is way beyond my current abilities) or fork bash sh. But it should been that complicated right.
I am open to suggestion where to start.
I am rusty with C, so I am reading again a lot of basic stuff.
With the xml file, later i was thinking on making a program to store this data and/or editing this data so i can create tutials and howto.
I can think of many ways of expanding this up to using printscreen so all the stored images go to a file you can upload to a server (for the moment i am glad to store the data). It could be a usefull tool.
ps. I do know this can be use for evil things too.
There already exists the script command, which will record all input and output into the terminal, writing it into a transcript. I would recommend just using that, unless you have particular needs that it doesn't meet. Actually, the nicest version of script that I've seen has been the NetBSD version, so you may want to look into that if the Linux version doesn't meet your needs.
If you would like to write it yourself, instead of using system, I would recommend that you use fork/exec to create a single shell process, which you copy all input and output into. To get an idea of how this works, I'd recommend looking at the source code for an existing version of script.
Most shells have a script built-in which will simply record the text in- and out- from the command line. Not quite what you're looking for... To my surprise script is not a built in, which means it is a model for building what you want.
The script command does almost what you want: it simply records the text in- and out- from the command line.
If you make your prompt distinctive (so that you can reliably tell the difference between shell commands and everything else) you can post-process the output of script to achieve your goals. Alternately you can hack script to get it to emit the XML you're looking for.
You can also try approaching this from a different angle. Instead of using a regular shell, connect to the machine using ssh or telnet and run your commands that way. Many ssh/telnet clients (PuTTY, for instance) have an option to log all console input and output during the session. You should be able to post-process this log to generate whatever type of logfile that you need.
Depending on your setup, you might not even have to use a second machine (you should be able to ssh into yourself).

Resources