Mssql encoding problem with special characters - sql-server

Question

We are using the node-mssql package to insert into and read out of our azure mssql database.
The database is very simple, because it is just used as a key/value cache, where the value can be quite long and also contain special characters.
DB Schema:
create table cache
(
id int identity
constraint cache_pk
primary key nonclustered,
cacheKey varchar(500),
ttl bigint,
value varchar(max)
)
go
create index cache_cacheKey_index
on cache (cacheKey desc)
go
create index cache_ttl_index
on cache (ttl)
go
For some reason, when I insert values into "value", some special characters are not treated well.
Dash – example
turns into:
Dash  example
I have seen the same thing happening with the french apostrophe.
I also tried to change the collation already, but that did not help.
Also tried it by using nvarchar(max) as column type.
This is the insert code (including the sql):
const result = await conn.request()
.input('key', mssql.VarChar, key)
.input('value', mssql.Text, value)
.input('ttl', mssql.BigInt, cacheEndTime)
.query`INSERT INTO cache (cacheKey, value, ttl) VALUES (#key, #value, #ttl)`;
Can you please help with a correct table structure or sql statement to make this work?

Answer 1

I'm not sure if can help you, but have you checked the collation of the table, the database, and the server? The collate have differents levels.

Answer 2

The answer for your question are in one of this items:
Server collation
Table collation
Field collation
Cast the insert text
For example, if you create nvarchar (I recommend if you have international scenario) field, cast the text insert like N'text to insert'.
It will work ;)

Answer 3

I have found the answer.
Like #RealSeik and #Larnu already stated it was probably not a problem with the database or the queries themselves, but rather an input problem.
I realized, that node-sass has a type for Unicode text, where they took care of casting it correctly.
So instead of mssql.Text I changed it to mssql.NText.
So now the insert command looks as follows:
const result = await conn.request()
.input('key', mssql.VarChar, key)
.input('value', mssql.NText, value)
.input('ttl', mssql.BigInt, cacheEndTime)
.query`INSERT INTO cache (cacheKey, value, ttl) VALUES (#key, #value, #ttl)`;
I have also added collations to my other scripts, for good measure as well.
(that alone did not help, but for good measure)
ALTER DATABASE MyDbName
COLLATE Latin1_General_100_CI_AI_SC_UTF8 ;
create table cache
(
id int identity
constraint cache_pk
primary key nonclustered,
cacheKey varchar(500) COLLATE Latin1_General_100_CI_AI_SC_UTF8,
ttl bigint,
value varchar(max) COLLATE Latin1_General_100_CI_AI_SC_UTF8
)
go
create index cache_cacheKey_index
on cache (cacheKey desc)
go
create index cache_ttl_index
on cache (ttl)
go