i have a small problem with BCP functionality in SQL Server 2012.
The things is:
im loading .jpg image (167KB in size) using below command:
INSERT [tabela_testowa] ( Data )
SELECT * FROM OPENROWSET (BULK N'C:\foty\ch6_MagicShop.jpg', SINGLE_BLOB) a;
and then im trying to export it back to disk using:
BCP "SELECT data FROM tabela_testowa WHERE ID = 1" queryout "C:\test\file.jpg" -T -n -d test
File gets saved on disk no problem, size is also 167 KB but.. it cant be opened like the original copy.
I dont know whatever some parameter is wrong in BCP export? Or maybe it gets corrupted at import stage?
Anyone had similiar problems?
Thank god, thanks to #user_0 answer and #user3494351's cryptic answer and comment and this ancient forum post I finally figured this out after several hours of banging my head against the wall.
The issue is that BCP likes to add an extra 8 bytes to the file by default. This corrupts the file and makes it unable to be opened if you just use the native -n flag.
However, BCP allows you to specify a format file as output that can allow you to tell it not to add the extra 8 bytes. So I have a table I created (to be used in a cursor) in SQL Server that only has ONE ROW and ONE COLUMN with my binary data. Table must exist when you run the first command.
In command line first you need to do this:
bcp MyDatabase.MySchema.MyTempTable format nul -T -n -f formatfile.fmt
This creates formatfile.fmt in the directory you are in. I did on E:\ drive. Here's what it looks like:
10.0
1
1 SQLBINARY 8 0 "" 1 MyColumn ""
That 8 right there is the variable that bcp says how many bytes to add to your file. It is the bastard that is corrupting your files. Change that sucker to a 0:
10.0
1
1 SQLBINARY 0 0 "" 1 MyColumn ""
Now just run your BCP script, drop the -n flag and include the -f flag:
bcp "SELECT MyColumn FROM MyDatabase.MySchema.MyTempTable" queryout "E:\MyOutputpath" -T -f E:\formatfile.fmt
BCP is adding informations to his file. Just few data, but you are not exporting just a jpg file.
You say 167 KB, but watch the real bytes, not the rounded dimension. There will be a difference.
You cannot export the image via BCP.
OK so i solved the issue.
Format file has to be added using -f and path to the file. It can be create by running bcp without any format and order it to save format file to disk. Then we can use this format file so its no longer needed to answer those questions, and file itself has no additional data and can be opened without problems
i have a small problem with BCP functionality in SQL Server 2012.
The things is:
im loading .jpg image (167KB in size) using below command:
INSERT [tabela_testowa] ( Data )
SELECT * FROM OPENROWSET (BULK N'C:\foty\ch6_MagicShop.jpg', SINGLE_BLOB) a;
and then im trying to export it back to disk using:
BCP "SELECT data FROM tabela_testowa WHERE ID = 1" queryout "C:\test\file.jpg" -T -n -d test
File gets saved on disk no problem, size is also 167 KB but.. it cant be opened like the original copy.
I dont know whatever some parameter is wrong in BCP export? Or maybe it gets corrupted at import stage?
Anyone had similiar problems?
Thank god, thanks to #user_0 answer and #user3494351's cryptic answer and comment and this ancient forum post I finally figured this out after several hours of banging my head against the wall.
The issue is that BCP likes to add an extra 8 bytes to the file by default. This corrupts the file and makes it unable to be opened if you just use the native -n flag.
However, BCP allows you to specify a format file as output that can allow you to tell it not to add the extra 8 bytes. So I have a table I created (to be used in a cursor) in SQL Server that only has ONE ROW and ONE COLUMN with my binary data. Table must exist when you run the first command.
In command line first you need to do this:
bcp MyDatabase.MySchema.MyTempTable format nul -T -n -f formatfile.fmt
This creates formatfile.fmt in the directory you are in. I did on E:\ drive. Here's what it looks like:
10.0
1
1 SQLBINARY 8 0 "" 1 MyColumn ""
That 8 right there is the variable that bcp says how many bytes to add to your file. It is the bastard that is corrupting your files. Change that sucker to a 0:
10.0
1
1 SQLBINARY 0 0 "" 1 MyColumn ""
Now just run your BCP script, drop the -n flag and include the -f flag:
bcp "SELECT MyColumn FROM MyDatabase.MySchema.MyTempTable" queryout "E:\MyOutputpath" -T -f E:\formatfile.fmt
BCP is adding informations to his file. Just few data, but you are not exporting just a jpg file.
You say 167 KB, but watch the real bytes, not the rounded dimension. There will be a difference.
You cannot export the image via BCP.
OK so i solved the issue.
Format file has to be added using -f and path to the file. It can be create by running bcp without any format and order it to save format file to disk. Then we can use this format file so its no longer needed to answer those questions, and file itself has no additional data and can be opened without problems
I'm trying to use a non-XML bcp format file to import data into LocalDB on Win7 64 bit. Simplest possible use case.
OS Name: Microsoft Windows 7 Home Premium
OS Version: 6.1.7601 Service Pack 1 Build 7601
LocalDB version: Microsoft SQL Server 2014 (12.0.2000.8)
BCP version: 12.0.2000.8
Basically, latest version of everything downloaded from Microsoft SQL Server 2014 site a few days ago.
I'm able to connect to the LocalDB instance via bcp to create a format file, but the generated format file doesn't work when trying to re-import the simplest possible data using it. No matter what I try, bcp loads zero rows, fails silently and prints no error information to the specified error file.
/* create the table */
use try_db;
create table try(num integer);
/* create the format file based on the table. */
bcp try_db.dbo.TRY format nul -n -T -f TRY.fmt -S (localdb)\default_db
/* above command creates a file TRY.fmt with the following contents */
12.0
1
1 SQLINT 1 4 "" 1 num ""
/* then I create a file data.txt, with just the number 99 in it, followed by a Windows line terminator (\r\n) */
/* then try importing the file into the table */
bcp try_db.dbo.TRY in data.txt -f TRY.fmt -T -S (localdb)\default_db -e errors.txt
Result:
Starting copy...
0 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 1
Nothing is written to errors.txt. I am just not able to get bcp to import anything at all using a format file!
I haven't tried it with SQL Server itself (only with LocalDB) but it shouldn't matter for stuff as simple as this.
I tried editing the TRY.fmt file line as follows:
1 SQLINT 1 4 "\r\n" 1 num ""
But that didn't help either.
I am able to get it to successfully import using -c instead of -f:
bcp try_db.dbo.TRY in data.txt -c -T -S (localdb)\default_db -e errors.txt
Any thoughts on (a) why bcp won't import using the format file, and (b) why it prints no errors to the specified error file? There must be something really simple I'm getting wrong here.
Please, no recommendations to use BULK INSERT or SSIS (etc) instead. bcp should just work as documented!
The format file is describing the source data not the destination. When you use -c or datafiletype='char' your input datatypes must be SQLCHAR. Native datatypes are only valid when using -n or datafiletype='native'. A source file in native format is always binary so bcp needs to know the data type of each field in order to read the correct amount of bytes and interpret them correctly.
I think I found the answer. The bcp format spec doesn't work properly! It seems that even for numeric or datetime import fields, you have to specify "SQLCHAR" as the datatype in the .fmt file. Any attempt to use the actual .fmt file generated by "bcp format" is hopeless -- if it gives you SQLINT or SQLDATE lines back, you have to replace those with SQLCHAR for the thing to work, even if the db columns are in fact numeric or date/datetime types.
What a crock!
i have a small problem with BCP functionality in SQL Server 2012.
The things is:
im loading .jpg image (167KB in size) using below command:
INSERT [tabela_testowa] ( Data )
SELECT * FROM OPENROWSET (BULK N'C:\foty\ch6_MagicShop.jpg', SINGLE_BLOB) a;
and then im trying to export it back to disk using:
BCP "SELECT data FROM tabela_testowa WHERE ID = 1" queryout "C:\test\file.jpg" -T -n -d test
File gets saved on disk no problem, size is also 167 KB but.. it cant be opened like the original copy.
I dont know whatever some parameter is wrong in BCP export? Or maybe it gets corrupted at import stage?
Anyone had similiar problems?
Thank god, thanks to #user_0 answer and #user3494351's cryptic answer and comment and this ancient forum post I finally figured this out after several hours of banging my head against the wall.
The issue is that BCP likes to add an extra 8 bytes to the file by default. This corrupts the file and makes it unable to be opened if you just use the native -n flag.
However, BCP allows you to specify a format file as output that can allow you to tell it not to add the extra 8 bytes. So I have a table I created (to be used in a cursor) in SQL Server that only has ONE ROW and ONE COLUMN with my binary data. Table must exist when you run the first command.
In command line first you need to do this:
bcp MyDatabase.MySchema.MyTempTable format nul -T -n -f formatfile.fmt
This creates formatfile.fmt in the directory you are in. I did on E:\ drive. Here's what it looks like:
10.0
1
1 SQLBINARY 8 0 "" 1 MyColumn ""
That 8 right there is the variable that bcp says how many bytes to add to your file. It is the bastard that is corrupting your files. Change that sucker to a 0:
10.0
1
1 SQLBINARY 0 0 "" 1 MyColumn ""
Now just run your BCP script, drop the -n flag and include the -f flag:
bcp "SELECT MyColumn FROM MyDatabase.MySchema.MyTempTable" queryout "E:\MyOutputpath" -T -f E:\formatfile.fmt
BCP is adding informations to his file. Just few data, but you are not exporting just a jpg file.
You say 167 KB, but watch the real bytes, not the rounded dimension. There will be a difference.
You cannot export the image via BCP.
OK so i solved the issue.
Format file has to be added using -f and path to the file. It can be create by running bcp without any format and order it to save format file to disk. Then we can use this format file so its no longer needed to answer those questions, and file itself has no additional data and can be opened without problems
I just rooted my Nexus 5 using this method: http://www.phonearena.com/news/How-to-root-Google-Nexus-5_id49014
I also enabled USB debugging in the developer options.
Then I tried to pull a database file from my device using this command:
adb pull /data/data/path.to.package/databases/data /sdcard/test
I get permission denied error.
I don't have the debugged flag set in that app I tried to access. Is that the reason I can't access that file? If yes, are there any workarounds to access an apps files?
You can use run-as shell command to access private application data.
If you only want to copy database you can use this snippet, provided in
https://stackoverflow.com/a/31504263/998157
adb -d shell "run-as com.example.test cat /data/data/com.example.test/databases/data.db" > data.db
I had the same problem. My work around is to use adb shell and su. Next, copy the file to /sdcard/Download
Then, I can use adb pull to get the file.
Did you try adb remount after giving adb root?
This generic solution should work on all rooted devices:
adb shell "su -c cat /data/data/com.android.providers.contacts/databases/contacts2.db" > contacts2.d
The command connects as shell, then executes cat as root and collects the output into a local file.
In opposite to #guest-418 s solution, one does not have to dig for the user in question.
Plus If you get greedy and want all the db's at once (eg. for backup)
for i in `adb shell "su -c find /data -name '*.db'"`; do
mkdir -p ".`dirname $i`"
adb shell "su -c cat $i" > ".$i"
done
This adds a mysteryous question mark to the end of the filename, but it is still readable.
If you get could not copy and permissions are right disable selinux.
Check if selinux is enabled.
$ adb shell
$su
# getenforce
Enforcing
Selinux is enabled and blocking/enforcing.
Disable selinux
# setenforce 0
do your stuff and set selinux to enforcing.
# setenforce 1
I had just the same problem, here's how to deal with it:
adb shell to the device
su
ls -l and check current access rights on the file you need. You'll need that later.
go to the file needed and: chmod 777 file.ext. Note: now you have a temporary security issue. You've just allowed all the rights to everyone! Consider adding just R for users.
open another console and: adb pull /path/to/file.ext c:\pc\path\to\file.exe
Important: after you're done, revert the access rights back to the previous value (point 3)
Someone mentioned something similar earlier.
Thanks for the comments below.
This answer ended up working for me: https://stackoverflow.com/a/15559278/53001
Backup to a file, pull the backup, and then convert it to a tarball and extract it.
adb backup -f myAndroidBackup.ab com.corp.appName
dd if=myAndroidBackup.ab bs=1 skip=24 | python -c "import zlib,sys;sys.stdout.write(zlib.decompress(sys.stdin.read()))" | tar -xvf -
$ adb shell
$su
# getenforce
Enforcing
now try
adb shell
su
chmod 777 /path/to/yout_file
exit from shell
open new CMD and try adb pull /path/to/yout_file
It will work fine now.
you'll have security problems because of this any application will be able to access your database.
#guest-418 tips works well:
adb -d shell "run-as com.example.test cat /data/data/com.example.test/databases/data.db" > data.db
However, if you want to use a GUI, use Android Studio's Device File Explorer.
Launch Android Studio
Click on Device File Explorer at bottom right-side
Navigate to your app's file:
/data/data/path.to.package/databases/data
Right-mouse click select Save As and save to a local folder
I have been having Android Monitor hang on me lately on macOS. Device File Explorer works well for this purpose.
Create a folder in sdcard :
adb shell "su 0 mkdir /sdcard/com.test"
Move your files to the new folder :
adb shell "su 0 mv -F /data/data/com.test/files/ /sdcard/com.test/"
You can now use adb pull :
adb pull /sdcard/com.test
When executing adb commands, by default, a limited privileges user is used, the same kind of limited privilege user that is assigned to an app, and you have requested for all privelages to use all features of the device.
This kind of limited user helps protect your phone from malware, by restricting the access between apps, and the system. This is the reason you are unable to access app data and system data on an unrooted phone. The act of rooting means becoming user 0, the super user of the system, capable of any action, and is the highest privilege. Your apps however, are still secure in that they can not talk to eachother.
Now when accessing secure files, note that you do not want to change the permissions of the file when you access it, which may allow for vulnerabilities.
An option that you could use instead, is to make a copy of the file on the sdcard as root, modify that as a standard user, and then move it back into the filesystem as root, while preserving the file permissions of the original file.
Since I've updated to Android Oreo, I had to use this script to fix 'permission denied' issue.
This script on Mac OS X will copy your db file to Desktop. Just change it to match your ADB_PATH, DESTINATION_PATH and PACKAGE NAME.
#!/bin/sh
ADB_PATH="/Users/xyz/Library/Android/sdk/platform-tools"
PACKAGE_NAME="com.example.android"
DB_NAME="default.realm"
DESTINATION_PATH="/Users/xyz/Desktop/${DB_NAME}"
NOT_PRESENT="List of devices attached"
ADB_FOUND=`${ADB_PATH}/adb devices | tail -2 | head -1 | cut -f 1 | sed 's/ *$//g'`
if [[ ${ADB_FOUND} == ${NOT_PRESENT} ]]; then
echo "Make sure a device is connected"
else
${ADB_PATH}/adb exec-out run-as ${PACKAGE_NAME} cat files/${DB_NAME} > ${DESTINATION_PATH}
fi
This is a bit late, but installing adbd Insecure worked for me. It makes adb run in root mode on production ("secure") devices, which is what you likely have.
A paid version is also available on Google Play if you want to support the developer.
I had a similar problem to yours on windows as the following.
D:\ProgramFiles\Android> adb pull /data/local/tmp/com.packagename_dumped_1766.dex D:\ProgramFiles\Android\com.packagename_dumped_1766.dex
adb: error: failed to copy '/data/local/tmp/com.packagename_dumped_1766.dex' to 'D:\ProgramFiles\Android\com.packagename_dumped_1766.dex': remote Permission denied
My solution:
At first I also made an attempt to use cat as ansi_lumen answered, but I got into trouble about CR and LR (\r\n) characters.
And then I just had to change those file permisions by chmod and pulled again to this problem was solved without introducing other problems. After that, may
we need to restore their original permissions as Goran Devs answered.
So just pay a little attention.
TL;DR
My story:
Firstly, I used the cat to download all files from android to my windows,
#echo off
cd /d %~dp0
:: %~dp0 = D:\ProgramFiles\Android\
SET ThisBatDir=%~dp0
:: adb shell ls /data/local/tmp/com.packagename_dumped_* > %~dp0\dump_file_list.txt
FOR /f "delims=" %%a in ('adb shell ls /data/local/tmp/com.packagename_dumped_*') do call :processline %%a %%~nxa
goto :eof
:: https://stackoverflow.com/questions/232651/why-the-system-cannot-find-the-batch-label-specified-is-thrown-even-if-label-e
:processline
SET RemoteFullPath=%1
set FileName=%2
:: echo "%RemoteFullPath%|%ThisBatDir%|%FileName%"
call adb shell su -c cat %RemoteFullPath% > %ThisBatDir%%FileName%
goto :eof
:eof
However, those downloaded dex files were broken because of CR and LR (\r\n) characters on windows.
We can use hexdump to inspect its content in Hex+ASCII form (or Notepad++ with "View > Show Symbol > Show All Characters" checked). Note, the 5th and 6th byte (0d 0a)).
ssfang#MONITO ~
$ hexdump -C -n32 /cygdrive/d/ProgramFiles/Android/com.packagename_dumped_1448.dex # a bad dex
00000000 64 65 78 0d 0d 0a 30 33 35 00 f7 8e e4 b5 03 c6 |dex...035.......|
00000010 29 22 98 55 21 e9 70 49 fe c8 e4 cc fa 94 cd 63 |)".U!.pI.......c|
00000020
ssfang#MONITO ~
$ hexdump -C -n32 /cygdrive/d/ProgramFiles/Android/classes.dex # a normal dex
00000000 64 65 78 0a 30 33 35 00 b5 73 03 3a 0b 9d a2 47 |dex.035..s.:...G|
00000010 a8 78 a4 f0 bb e1 64 3f e5 b9 cb a0 bd 1b e2 71 |.x....d?.......q|
00000020
Versions
adb version // to check adb client version in your desktop
adb shell adbd --version // to check adbd's version in your Android. Please note that some users reported error with this if executed without root access.
D:\ProgramFiles\Android>adb version
Android Debug Bridge version 1.0.41
Version 29.0.6-6198805
Installed as D:\ProgramFiles\Android\Sdk\platform-tools\adb.exe
D:\ProgramFiles\Android>adb shell adb version
Android Debug Bridge version 1.0.32
Even if restarting adbd as root, it was still the shell user after .
D:\ProgramFiles\Android> adb root
restarting adbd as root
D:\ProgramFiles\Android> adb shell id
uid=2000(shell) gid=2000(shell) groups=1003(graphics),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats) context=u:r:shell:s0
So I first viewed its file permision,
D:\ProgramFiles\Android> adb shell ls -l /data/local/tmp
-rwsr-sr-x shell shell 589588 2017-09-14 15:08 android_server
-rwsr-sr-x shell shell 1243456 2017-09-14 15:08 android_server64
-rw-rw-rw- shell shell 1536 2020-03-28 17:15 com.packagename.tar.gz
-rw-r----- root root 57344 2020-03-28 17:45 com.packagename_dumped_1766.dex
drwxrwxr-x shell shell 2018-08-12 09:48 device-explorer
-rwsrwsr-x shell shell 13592 2019-02-04 17:44 drizzleDumper
-rwxrwxrwx shell shell 5512504 2018-05-06 01:27 lldb-server
-rwxr-xr-x shell shell 12808 2020-03-26 22:16 mprop
then, changed its permision,
D:\ProgramFiles\Android> adb shell su -c chmod 777 /data/local/tmp/com.packagename_dumped_*
D:\ProgramFiles\Android> adb shell ls -l /data/local/tmp
-rwxrwxrwx root root 57344 2020-03-28 17:45 com.packagename_dumped_1766.dex
As a result, I made it.
D:\ProgramFiles\Android> adb pull /data/local/tmp/com.packagename_dumped_1766.dex D:\ProgramFiles\Android\com.packagename_dumped_1766.dex
/data/local/tmp/com.packagename_dumped_1766.de... 1 file pulled, 0 skipped. 3.6 MB/s (57344 bytes in 0.015s)
Now, jadx-gui-dev.exe or sh d2j-dex2jar.sh -f ~/path/to/apk_to_decompile.apk could properly enjoy them.
The pull command is:
adb pull source dest
When you write:
adb pull /data/data/path.to.package/databases/data /sdcard/test
It means that you'll pull from /data/data/path.to.package/databases/data and you'll copy it to /sdcard/test, but the destination MUST be a local directory. You may write C:\Users\YourName\temp instead.
For example:
adb pull /data/data/path.to.package/databases/data c:\Users\YourName\temp