Anyway I can use the command line or CnosQL to find out the storage size of each CnosDB database, as well as the measurement ones?
I have to use following workaround so far according to the doc:
du -sh /var/lib/cnosdb/data/SomeDatabaseName
Related
I am using apache-jena-4.5.0 and fuseki pretty much out-of-the-box. I had created a TDB2 dataset using fuseki, but now shut it off and using command-line utilities of jena on a Windows box inside a bash shell.
My basic command is:
java -cp "*" tdb2.tdbquery --loc ~/path/to/databases/DEMO--explain --set arq:logExec=FINE --time --query ~/path/to/demoquery.txt
And my question is why does the output always contain only Reorder/generic like this:
15:56:00 INFO exec :: Reorder/generic
Even after I have tried all these:
successfully run tdb2.tdbstats and gotten a reasonable-looking temp.opt file as output
moved that temp.opt to each of /path/to/DEMO/stats.opt and /path/to/DEMO/Data-001/stats.opt
tried uppercase STATS.OPT for each since I'm on windows, just to be sure
Still I don't seem to be able to produce any output with Reorder/stats
This question did not contain enough detail to answer. The intended question was why won't TDB2 optimize my query and the answer was in the SPARQL, not in the invocation of tdb2.tdbquery or the location of the stats.opt file.
My SPARQL contained multiple FROM clauses, which forced TDB into BGP mode (instead of quads) and thwarted any optimization. As best we can tell at the moment, one wishing to use the TDB2 optimizer should use either the default graph, or a combination of FROM NAMED and GRAPH which causes the evaluation of graphs one at a time.
I know I can get mount point utilization using command line tools like df and doing something like the following:
popen("df -h /var/log | awk '{if($1==\"tmpfs\") print $5}'","r")
where the fifth column is the usage percentage.
What I want to know is if there is a way to access mount point utilization from /proc/, such as /proc/mounts. I feel that getting this information from /proc/ would be a quicker and more efficient solution than using df and awk to parse out the usage. I've searched mntent.h but couldn't find anything promising there either. Any ideas?
You probably want the statfs(2) syscall (it does not use /proc/). You probably would use it thru statvfs(3) Posix function.
Notice that the /proc/ filesystem (and also /sys/ ...) contains pseudo-files which are generated by the kernel on demand without any IO. See proc(5). You could read sequentially /proc/self/mounts and/or /proc/self/mountstats and/or appropriate files under /proc/fs/ and/or /sys/ (such as some files in /proc/fs/ext4/sda1/ or even in /sys/block/sda/sda1/ for my desktop computer ; it would be different on yours ....)
Perhaps systemd is also able to give such information, but I don't know it enough.
I want to transfer a test file to mainframe, but the test file has lines exceeding 80 characters, which is default for FTP. Because the created dataset has record length 80, I am getting
451-File transfer failed. File contains records that are longer than the LRECL of the new file.
error. I tried this;
curl --ftp-ssl -k -v -B -T BBBBB -u USERNAME:PASS ftp://HOST_NAME:PORT/'DATASET_NAME(BBBBB)'
To solve this problem, I added -Q "site lrecl=250" but this didn' t help.
Are you creating a dataset or are creating a Member in a PDS ???. The syntax DATASET_NAME(BBBBB) implies you could be creating a member in an existing PDS.
The LRECL characteristics are defined in the PDS definition and can not be changed by the send command.
If it is an existing PDS, you will need to create a new dataset / PDS either through the send command or be creating a new dataset on the Mainframe with the correct characteristics and then doing the send.
Pre-allocate, on the Mainframe, a dataset with the characteristics that you want, like the LRECL (record-length) and RECFM (record format, are all the records a "fixed" length, or can they differ?).
If you ftp to that dataset, does that give you a problem?
I don't think that 80 is a "default" value for ftp, it is likely just the LRECL of the dataset you are trying to stuff the data into.
Somewhere amongst the Mainframe Support staff are people who know the standards for your Mainframe's ftp usage. It would be worth locating them, explaining what you have and what you need to do, and ask them the correct way to do it. Better than struggling away now, then have it "bounced" along the way as being "non-standard".
If this is a one-time or seldom repeated task, you could transfer the file to the Unix file system and then use OGET to create the "classic" z/OS file.
I'm trying MongoDB but I don't understand the files created by it.
If I create a document on my db collection lets say users, it will create these files on my /data/db/
_tmp (folder)
mongod.lock
users.0 - 16.8MB
users.1 - 33.6MB
users.ns - 16.8MB
I've added some documents and the files size haven't changed..I'm a bit lost here...does anyone know how those files work? I've tried to open them with gedit/vim and nothing.
Thanks in advance for any help.
The .0, .1 files are datafiles. Each datafile is preallocated to a particular size. (This is done to prevent file system fragmentation, among other reasons.) The first filename for a database is .0, then .1, etc. .0 will be 64MB, .1 128MB, et cetera, up to 2GB. Once the files reach 2GB in size, each successive file is also 2GB.
Information regarding datafiles can be found here:
http://www.mongodb.org/display/DOCS/Excessive+Disk+Space
The ".ns" files are namespace files. Each collection and index would count as a namespace. Each namespace is 628 bytes, the .ns file is 16MB by default.
Thus if every collection had one index, we can create up to 12,000 collections. The --nssize parameter allows you to increase this limit.
Maximum .ns file size is 2GB.
Mongo pre-allocates space in chunks, so your file size will stay at 16Mb until you have more than 16Mb of data, and it rolls over a new chunk.
I'm not sure if it was asked, but I couldn't find anything like this.
My program uses a simple .txt file for log purposes, It just creates/opens a file and appends lines.
After some time, I started to log quite a lot of activities, so the file became too large and hardly readable. I know, that it's not write way to do this, but I simply need to have a readable file.
So I thought maybe there's a simple file format for log files and a soft to view it or if you'd have any other suggestions on this question?
Thanks for help in advance.
UPDATE:
It's access 97 application. I'm logging some activities like Form Loading, SELECT/INSERT/UPDATE to MS SQL Server ...
The log file isn't really big, I just write the duration of operations, so I need a simple way to do this.
The Log file is created on a user's machine. It's used for monitoring purposes logging some activities' durations.
Is there a way of viewing that kind of simple Log file highlighted with an existing tool?
Simply, I'd like to:
1) Write smth like "'CurrentTime' 'ActivityName' 'Duration in milliseconds' " (maybe some additional information like query string) into a file.
2) Open it with a tool and view it highlighted or somehow more readable.
ANSWER: I've found a nice tool to do all I've wanted. Check my answer.
LogExpert
The 3 W's :
When, what and where.
For viewing something like multitail ("tail on steroids") http://www.vanheusden.com/multitail/
or for pure ms windows try mtail http://ophilipp.free.fr/op_tail.htm
And to keep your files readable, you might want to start new files when if the filesize of the current log file is over certain limit. Example:
activity0.log (1 mb)
activity1.log (1 mb)
activity2.log (1 mb)
activity3.log (1 mb)
activity4.log (205 bytes)
A fairly standard way to deal with logging from an application into a plain text file is to:
split the logs into different program functional areas.
rotate the logs on a daily/weekly basis (i.e. split the log on a size or date basis)
so the current log is "mylog.log" or whatever, and yesterday's was "mylog.log.1" or "mylog.ddmmyyyy.log"
This keeps the size of the active log manageable. And then you can just have expiry rules so that old logs get thrown away on a regular basis.
In addition it would be a good idea to have different log levels for your application (info/warning/error/fatal) so that you're not logging more than is necessary.
First, check you're only logging things that are useful.
If it's all useful, make sure it is easily parsable by tools such as grep, that way you can find the info you want. Make sure you have the type of log entry, the date/time all conforming to a layout.
Build yourself a few scripts to extract the information for you.
Alternatively, use separate log files for different types of entries.
Basically you better just split logs according to severity. You'll rarely need to read all logs for the whole system. For example apache allows to configure error log and access log, pretty obvious what info exactly they have.
If you're under linux system grep is your best tool to search through logs for specific entries.
Look at popular logfiles like /var/log/syslog on Unix to get ideas:
MMM DD HH:MM:SS hostname process[pid]: message
Example out of my syslog:
May 11 12:58:39 raphaelm anacron[1086]: Normal exit (1 job run)
But to give you the perfect answer we'd need more information about what you are logging, how much and how you want to read the logs.
If only the size of the log file is the problem, I recommend using logrotate or something similar. logrotate watches log files and, depending on how you configured it, after a given time or when the log file exceeds a given size, it moves the log file to an archive directory and optionally compresses it. Then the original log file is truncated. For example, you could configure it to archive the log file every 24 hours or whenever the files size exceeds 500kb.
If this is a program, you might investigate apache logging libraries (http://logging.apache.org/) Out of the box, they'll give you a decent logging format out of the box. They're also customizable, so you can simplify your parsing job.
If this is a script, see some of the other answers.
LogExpert
I've found it here. Filter is better, than in mtail. There's an option of highlighting just adding a string and the app is nice and readable. You can customize columns as you like.