I am trying to hash a string in postgresql using sha256 as follow:
select digest('This is a test','sha256');
this returns below hashed value:
\xc7be1ed902fb8dd4d48997c6452f5d7e509fbcdbe2808b16bcf4edce4c07d14e
Now i want to retrieve my initial string using this hashed value but unable to find anything in the postgres docs regarding it, Any help regarding it will be appreciable.
There is a difference between hashing and encryption:
an encrypted value can be descrypted to get the original back, so encryption is loss-free and two different clear text values will always result in different encrypted values
a hash cannot be decrypted, because information is lost; different values can result in the same hash, although it is desirable that these "collisions" do not happen too often
Hashing is irreversible, while encryption is reversible.
Now digest is a hashing function:
digest(data text, type text) returns bytea
digest(data bytea, type text) returns bytea
Computes a binary hash of the given data.
So you won't be able to recover the original string.
You can use pgcrypto extension in Postgresql to save data after encryption.
For insertion,
INSERT INTO tablename (columnname1, columnname2) VALUES (PGP_SYM_ENCRYPT('value1', 'aes_key'), PGP_SYM_ENCRYPT('value2', 'aes_key'));
For fetching,
SELECT PGP_SYM_DECRYPT(columnname1::bytea, 'aes_key') as columnname1, PGP_SYM_DECRYPT(columnname2::bytea, 'aes_key') as columnname2 from tablename;
Related
I am computing HASH value of each row in a table (for diffing purpose), after implementing the algorithm I am testing the results.
Results are consistent and algorithm somewhat seems to work, but testing it step by step I found a strange result.
The script:
SELECT HASHBYTES('SHA1', (SELECT INNERTBL.VALUT FOR XML RAW)) as KHASH
FROM ACLING AS INNERTBL
Should perform the SHA1 calculation on the table key, but when I perform the same calculation with external tool I get different results:
In fact when I perform SHA1('<row VALUT="A"/>') with external tool (tool here: https://emn178.github.io/online-tools/sha1.html) I get a different result:
So my question is, there is something wrong with my logic or simply SQL Server use some non standard SHA1 "parametrization"? (I have suspect about the use of a, may be standard but particular, padding scheme)
Example in fiddler: https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=efa4e0ba11c112f54e36afb5d54d2cce
SELECT HASHBYTES('SHA1','<row VALUT="A"/>'), --- you are testing this
HASHBYTES('SHA1',N'<row VALUT="A"/>') -- ..but for xml returns Nvarchar
It is important to remember that you get the same result only if the string are binary the same. For example if the two strings uses different characterset they will have different HASH value. For more details please check thsi out https://security.stackexchange.com/questions/18290/is-sha-1-hash-always-the-same
After testing and ensuring my Send_email function authenticates successfully using the hardcoded user and password in the SQL Table, I am now trying to Hash the password.
the way my columns are set up is something like this:
variable | value
password | someP#ssword
the columns are varchar both, with the value column being 1000 length (is that too much? i set it to that much cause i read that sha 512 requires a lot of length or something, and seen examples using 1000 at least so went with that)
I am using this query to hash,
INSERT INTO [dbo].[Table] (value)
VALUES(HASHBYTES('SHA2_512', 'someP#ssword'))
but it generates a nonbinary hash, which i am suspecting is why i am receiving this error of email authentication failed because it probably cant decipher the nonbinary password characters. but the problem is i have other values in the value column, so i can't convert the whole column to varbinary.
so is there a way to hash the password that is hardcoded or i have to insert it as hash? or is there a way i can convert just that particular field/cell without having to alter the rest of the design of the value column so it wont affect other values in there as well? or am i supposed to create a completely separate column for password and set it to binary?
EDIT: I have to pass the password in this email function call for authentication:
Send-EMail -EmailFrom $From_Email -SendTo $To_Email -Body $Email_Body -Subject $Job_Success_Email_Subject -Username $SmtpUser -Password $SmtpPassword
but Von in the comment said i can't pass the hashed password in there as credential. then that means i have to keep it nonhashed in the table? i thought hashing would work perfectly in this situation...
It looks like you have been confused by the irrelevant discussion in the comments above.
First of all: hashed password would not work in your Send-EMail function as the function has no way of "unhashing" said password. Read this introduction to Hashing vs Encryption.
If you want to secure your password and be able to retrieve original value you will need to encrypt it. The topic of encryption is quite large and way outside the scope of what can be written in SO. I will provide a few links for you to read:
http://www.manjuke.com/2018/03/data-encryption-in-sql-server-using-t.html
https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/encrypt-a-column-of-data?view=sql-server-2017
https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/sql-server-and-database-encryption-keys-database-engine?view=sql-server-2017
Encryption by pass phrase would be the simplest to implement but also weakest as anyone reading the code of an SP will find out the pass phrase and therefore can decrypt data. Note that pass phrase itself can be passed into ENCRYPTBYPASSPHRASE as a parameter, allowing you to store it (securely) elsewhere outside the database i.e. you don't have to hard code in your SP code. You will need to implement pass phrase storage method yourself if you decide to go this way.
Encryption using keys and certificates offers a very secure method but requires some time to set-up in addition to very carefully backing up your keys. If you loose your keys your data is gone i.e. you can never decrypt it.
As far as storing binary data in varchar column goes - easy, here is an example:
DECLARE #BinValue VARBINARY( 500 ) = HASHBYTES('SHA2_512', 'someP#ssword')
DECLARE #StringBinValue VARCHAR( 500 ) = CONVERT( VARCHAR( 500 ), #BinValue, 1 )
SELECT #BinValue, #StringBinValue, CONVERT( VARBINARY( 500 ), #StringBinValue, 1 ) AS BackToString
I have used your original HASHBYTES function as an example but you will need to change it to an encryption function.
Hope this clarifies things for you.
How to encrypt column in postgres database using pgcrypto addon ?
I am using postgres 9.3 and i need to encrypt one of my column , does postgres also support Aes encryption or by any mean i can achieve it ?
Yes, Postgres pgcrypto module does support AES. All details with examples can be found here. As for the sample usage:
-- add extension
CREATE EXTENSION pgcrypto;
-- sample DDL
CREATE TABLE test_encrypt(
value TEXT
);
INSERT INTO test_encrypt VALUES ('testvalue');
-- encrypt value
WITH encrypted_data AS (
SELECT crypt('PasswordToEncrypt0',gen_salt('md5')) as hashed_value
)
UPDATE test_encrypt SET value = (SELECT hashed_value FROM encrypted_data);
Validate password:
SELECT (value = crypt('PasswordToEncrypt0', value)) AS match FROM test_encrypt;
Returns:
match
-------
t
(1 row)
The above is not encryption as encryption is reversible which means if you encrypt some secret text or value, you should be able to know what that secret value or text was unlike hashing where you want to verify if the user-provided value matches the hashed value or not.
This is how you would encrypt column data using pgcrypto module.
create extension if not exists pgcrypto; -- this will install the module if not installed
CREATE TABLE agents (
id serial primary key,
name varchar not null
);
INSERT INTO agents (name) values
(pgp_sym_encrypt('Johny Smith', 'longsecretencryptionkey')),
(pgp_sym_encrypt('Bob Marley', 'longsecretencryptionkey'));
longsecretencryptionkey
is your encryption key. You can generate encryption key from here encryption key generator and choose the bit of your choice. The recommendation would be to choose min 256 bit.
Remember to keep the encryption key somewhere safe saved somewhere. If you lose your encryption key, you will not be able to decrypt it anymore. This is very crucial to understand.
This is how you would query them
SELECT pgp_sym_decrypt(name::bytea, 'longsecretencryptionkey') FROM users WHERE pgp_sym_decrypt(name::bytea, 'longsecretencryptionkey') ILIKE 'johny%'; -- querying for agents whose name start with johny
You can checkout this blog article which helped me
https://blog.andreiavram.ro/encrypt-postgresql-column/
Note:
Instead of keeping the name column in varchar, you can use bytea data type for the column. So the table becomes
CREATE TABLE agents (
id serial primary key,
name bytea not null
);
So, when querying you don't need to cast every time, you can do just this
SELECT pgp_sym_decrypt(name, 'longsecretencryptionkey') FROM users WHERE pgp_sym_decrypt(name::bytea, 'longsecretencryptionkey') ILIKE 'johny%'; -- querying for agents whose name start with johny
edit queue is full,
just put some description about crypt here:
一. crypt()
Password Hashing Functions
https://www.postgresql.org/docs/current/pgcrypto.html#id-1.11.7.35.7
The functions crypt() and gen_salt() are specifically designed for hashing passwords.
crypt() does the hashing and gen_salt() prepares algorithm parameters for it.
crypt(password text, salt text) returns text
Calculates a crypt(3)-style hash of password. When storing a new password, you need to use gen_salt() to generate a new salt value. To check a password, pass the stored hash value as salt, and test whether the result matches the stored value.
man 3 crypt
https://manpages.ubuntu.com/manpages/jammy/en/man3/crypt.3.html
The crypt, crypt_r, crypt_rn, and crypt_ra functions irreversibly “hash” phrase for storage in the system password database (shadow(5)) using a cryptographic “hashing method.”
二. pgp_sym_encrypt
PGP Encryption Functions
https://www.postgresql.org/docs/current/pgcrypto.html#id-1.11.7.35.8
The functions here implement the encryption part of the OpenPGP (RFC 4880) standard. Supported are both symmetric-key and public-key encryption.
pgp_sym_encrypt(data text, psw text [, options text ]) returns bytea
Encrypt data with a symmetric PGP key psw.
pgp_sym_decrypt(msg bytea, psw text [, options text ]) returns text
Decrypt a symmetric-key-encrypted PGP message.
In a fulltextsearch where the field is encrypted, is there a way to search for its content?
E.g.:
value on the database is name = "John Stewart" (encrypted)
I want to search for either records that contains "John" or "Stewart".
If encrypted properly this would be nearly impossible. I'm not aware of any options in MySQL that you could use to encrypt a column like you are suggesting. So you'd need to get all results, decrypt in your application, and check against the decrypted value.
If the value has instead been hashed it would be literally impossible, as the shorter string and the longer string would have completely independent hashes, and you wouldn't be able to turn the hash of the longer string back into the string itself to check.
I'm playing around with Asymmetric Encryption and Decryption, but I don't understand the results I'm getting when I try to decrypt values.
Why does this:
CREATE ASYMMETRIC KEY myasymkey
WITH ALGORITHM = RSA_2048
ENCRYPTION BY PASSWORD = '123pass!';
GO
SELECT DECRYPTBYASYMKEY(ASYMKEY_ID('myasymkey'),
EncryptByAsymKey(AsymKey_ID('myasymkey'), 'Greg'),
N'123pass!');
GO
Produce 0x47726567? I was expecting it to be Greg.
UPDATE: I'm dumb, 0x47726567 is Greg when converted from varbinary.
It's right - when you encrypt something it's treated as a byte array and is returned as such. 0x47 is G, 72 is r etc.
If you examine the documentation for DecryptByAsmKey you'll notice the return type is varbinary with a maximum size of 8,000 bytes. You'll also notice the convert in the example.
So if you are encrypting and decrypting strings you must convert like so
SELECT CONVERT(varchar(max),DECRYPTBYASYMKEY(ASYMKEY_ID('myasymkey'),
EncryptByAsymKey(AsymKey_ID('myasymkey'),
'Greg'),
N'123pass!'));
Also note you need to make sure you're converting to varchar(max) or nvarchar(max) depending on your input. If you tried
SELECT CONVERT(nvarchar(max),DECRYPTBYASYMKEY(ASYMKEY_ID('myasymkey'),
EncryptByAsymKey(AsymKey_ID('myasymkey'),
'Greg'),
N'123pass!'));
it would be wrong, as your input of 'Greg' is a varchar.
I think 0x47726567 is Greg (in ASCII).
All the encryption functions return varbinary values. You will have to cast the result as varchar.