I have built a tdengine cluster with three nodes. And want to the space that the cluster used. Are there any commands or sqls that can help me check how much space I have used? Just like hdfs using a command like
hadoop fs -du -h /warehouse/hive/warehouse/
using which I can get the space info for every path or an individual file.
or any sqls like
select segment_name, bytes from user_segments where segment_type = 'TABLE';
Select Tablespace_Name,Sum(bytes)/1024/1024 From Dba_Segments Group By Tablespace_Nameļ¼
these ORACLE sqls will return the physical space allocated to an individual table or the usage situation of the tablespace.
you can try this:
select last(disk_used,disk_total) from log.dn group by fqdn;
Related
I have Oracle Database 12c in a docker container. At one point, the space in the tablespace expanded (the users01.dbf file became 32GB in size) and I made a new file for this tablespace - users02.dbf. I analyzed the tables and indexes that occupy the most space and made them truncate. I see that the size of the largest tables and indexes has decreased, but the users01.dbf and users02.dbf files remain the same size:
users01.dbf 32G
users02.dbf 8.8G
Here is a screenshot:
Left size before, right after truncate command. How can I clean or reduce the size of users01.dbf and users02.dbf files and not break the database.
I am using the one query to find out the actual size that can be shrunk from the data file.
I am using it for almost 4 years and I don't know from where I got it but Yes, I got that script from one of the good blogs of the oracle.
Following script will generate the series of command which can be executed on DB to shrunk the data file or we can say reclaim the free space from the data file (only if more than 1 MB can be reclaimed) to perfect size without any error.
set linesize 1000 pagesize 0 feedback off trimspool on
with
hwm as (
-- get highest block id from each datafiles ( from x$ktfbue as we don't need all joins from dba_extents )
select /*+ materialize */ ktfbuesegtsn ts#,ktfbuefno relative_fno,max(ktfbuebno+ktfbueblks-1) hwm_blocks
from sys.x$ktfbue group by ktfbuefno,ktfbuesegtsn
),
hwmts as (
-- join ts# with tablespace_name
select name tablespace_name,relative_fno,hwm_blocks
from hwm join v$tablespace using(ts#)
),
hwmdf as (
-- join with datafiles, put 5M minimum for datafiles with no extents
select file_name,nvl(hwm_blocks*(bytes/blocks),5*1024*1024) hwm_bytes,bytes,autoextensible,maxbytes
from hwmts right join dba_data_files using(tablespace_name,relative_fno)
)
select
case when autoextensible='YES' and maxbytes>=bytes
then -- we generate resize statements only if autoextensible can grow back to current size
'/* reclaim '||to_char(ceil((bytes-hwm_bytes)/1024/1024),999999)
||'M from '||to_char(ceil(bytes/1024/1024),999999)||'M */ '
||'alter database datafile '''||file_name||''' resize '||ceil(hwm_bytes/1024/1024)||'M;'
else -- generate only a comment when autoextensible is off
'/* reclaim '||to_char(ceil((bytes-hwm_bytes)/1024/1024),999999)
||'M from '||to_char(ceil(bytes/1024/1024),999999)
||'M after setting autoextensible maxsize higher than current size for file '
|| file_name||' */'
end SQL
from hwmdf
where
bytes-hwm_bytes>1024*1024 -- resize only if at least 1MB can be reclaimed
order by bytes-hwm_bytes desc
/
It will generate the commands something like the following:
/* reclaim 1934M from 2048M */ alter database datafile 'C:\APP\TEJASH\VIRTUAL\ORADATA\ORCL\DATAFILE\TEJASH_DATAFILE_01.DBF' resize 115M;
/* reclaim 158M from 200M */ alter database datafile 'C:\APP\TEJASH\VIRTUAL\ORADATA\ORCL\DATAFILE\UNDO_DF_02.DBF' resize 43M;
/* reclaim 59M from 1060M */ alter database datafile 'C:\APP\TEJASH\VIRTUAL\ORADATA\ORCL\DATAFILE\O1_MF_SYSAUX_G9K5LYTT_.DBF' resize 1002M;
/* reclaim 3M from 840M */ alter database datafile 'C:\APP\TEJASH\VIRTUAL\ORADATA\ORCL\DATAFILE\O1_MF_SYSTEM_G9K5KK2J_.DBF' resize 838M;
You can directly execute all of them and you do not have to worry about calculating anything by yourself.
Please note that this script will work on the data files for which autoextensible is ON
I hope this will help you out.
Cheers!!
alter database datafile 'path_to_datafile/users01.dbf' resize 150M;
Repeat same for 02. Make sure path and file names are correct.
Since you truncated tables it should have dropped the segments altogether, so file should be able to shrink. If you get ORA-03297: file contains used data beyond requested RESIZE value then it means you have data at 150M mark so you should try increasing resize limit more until error goes away.
As always you shouldn't be doing stuff directly on Production but test it out.
I want to benchmark an HBase using YCSB. It's my first time using either.
I've gone through some online tutorials, and now I need to create a sample table of size 5 GB. But I don't know how to:
Batch-put a bunch of data into a table
Control the size to be around 5 GB
Could anyone give me some help on that?
Before, I've used HBase performance evaluation tool to load data into HBase. May be it can help you.
hbase org.apache.hadoop.hbase.PerformanceEvaluation
Various options are available for this tool. For your case you can set the data size to be 5GB.
This is pretty easy, the default (core) workload uses strings that are ~1KB each. So to get 5GB, just use 5,000,000 records.
You can do this by specifying the recordcount parameter in the command line, or creating your own workload file with this parameter inside.
Here's how you would do it on the command line (using the included workload workloada):
./bin/ycsb load hbase12 -P workloads/workloada -p recordcount=5000000
A custom file would look like this:
recordcount=5000000
operationcount=1000000
workload=com.yahoo.ycsb.workloads.CoreWorkload
readproportion=0.8
updateproportion=0.2
scanproportion=0
insertproportion=0
And then you just run:
./bin/ycsb load hbase12 -P myWorkload
This will insert all the data into your database.
So I have the following SOQL query that includes the ActivityHistories relationship of the Account object:
SELECT Id, Name, ParentId, (SELECT Description FROM ActivityHistories)
FROM Account
WHERE Name = '<some client>'
This query works just in in SOQLXplorer and returns 5 nested rows under the ActivityHistories key. In Talend, I am following the instructions from this page to access the sub-objects (although the example uses the query "up" syntax, not the query "down" syntax. My schema mapping is as follows:
The query returns the parent Account rows but not the ActivityHistory rows that are in the subquery:
Starting job GetActivities at 15:43 22/06/2016.
[statistics] connecting to socket on port XXXX
[statistics] connected
0X16000X00fQd61AAC|REI||
[statistics] disconnected
Job GetActivities ended at 15:43 22/06/2016. [exit code=0]
Is it possible to reference the subrows using Talend? If so, what is the syntax for the schema to do so? If not, how can I unpack this data in some ay to get to the Description fields for each Account? Any help is much appreciated.
Update: I have written a small python script to extract the ActivityHistory records and dump them in a file, then used a tFileInput to ingest the CSV and then continue through my process. But this seems very kludgey. Any better options out there?
I've done some debugging from the code perspective and it seems that if you specify correct column name, you will achieve the correct response. For your example, it should be: Account_ActivityHistories_records_Description
The output from tLogRow will be similar to:
00124000009gSHvAAM|Account1|tests;Lalalala
As you can see, the Description from all child elements are stored as 1 string, delimited by the semicolon. You can change the delimiter on Advanced Settings view on the SalesforceInput.
I have written a small python script (source gist here) to extract the ActivityHistory records and dump them in a file (command line argument), then used a tFileInput to ingest the CSV and then continue through my process.
I need to load data from sybase(production database) to HDFS. By using sqoop it is taking very long time and frequently hit the production database. So, I am thinking to create data files from sybase dump and after that copy the data files to hdfs. Is there any tool(open source) is available to create required data files(flat files) from sybase dump.
Thanks,
The iq_bcp command line utility is designed to do this on a per table basis. You just need to generate a list of tables, and you can iterate through the list.
iq_bcp [ [ database_name. ] owner. ] table_name { in | out } datafile
iq_bcp MyDB..MyTable out MyTable.csv -c -t#$#
-c specifies a character (plaintext) output
-t allows you to customize the column delimiter. You will want to use a character or series of characters that do not appear in your extact e.g. if you have a text column that contains text with a comma, a csv will be tricky to import without additional work.
Sybase IQ: iq_bcp
I need to export only subset of columns from a very large table containing large number of columns.Also this table contains million of rows so i want to export only specific rows from this table.
I have recently started using Mysql earlier i was working on Oracle.
This worked for me:
mysql -u USERNAME --password=PASSWORD --database=DATABASE \
--execute='SELECT `field_1`, `field_2` FROM `table_name`' -X > file.xml
And then importing the file, using command:
LOAD XML LOCAL INFILE '/pathtofile/file.xml'
INTO TABLE table_name(field_1, field_2, ...);
What format do you need the data in? You could get a CSV using a query. For example
SELECT column1,column2,column3,... FROM table WHERE column1=criteria1,....
INTO OUTFILE '/tmp/output.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
http://www.tech-recipes.com/rx/1475/save-mysql-query-results-into-a-text-or-csv-file/
An administration tool like phpMyAdmin (http://www.phpmyadmin.net/) could also be used to run the query and then export the results in a variety of formats.