When I save a string with blank spaces at the end, the string is being saved without the blank spaces.
In my development environment it works fine, but in production it doesn't.
Is there a configuration that can be done to force the blank spaces? It must be a configuration made via code.
As said by Damien, I will store data in its most natural representation and will format later.
Thanks!
Related
After importing some data into a Sql 2014 database, I realized that there are some fields in which the data replaced German characters such as (ü, ß, ä, ö, etc) with some weird characters. Ex.
München should be München
ChiemgaustraÃe should be Chiemgaustraße
Königstr should be Königstr
I would like to replace these characters with the right German letter. Ex.
ü -> ü
à - > ß
ö -> ö
However when I run queries like the following to try to identify which rows have these characters, the queries returns 0 rows.
select address
from Directory
where street like N'%ChiemgaustraÃe 50%'
select address
from Directory
where street like N'%ü%'
Is there a query I can run to identify and replace these characters?
I must clarify that most of the data was imported correctly, in fact I believe the strange characters were already part of the original data.
Also, I think I can export the data to a text file, replace the characters and re-import, but I was wondering if there is a way to do it directly in sql.
Thanks in advance for the help.
I couldn't get it fix using only sql.
FutbolFa suggestion worked for the most part but there were a couple of symbols, in particular "Ã" that wasn't picked up by any query a tried. I ended up exporting the data to a text file and replacing the symbols there. Then I just re-imported the info.
I've saved all my .php files as utf-8, and everything works fine. whenever I enter certain characters, such as å and ë, it works just fine. however, with the data is retrieved from the database through a mysqli query, and then put into the $page variable which eventually is echo'ed at the end of the document, it displays a ? where there should be a ë. when I make an empty page, with only the dbconnect, the query, and make that echo nothing but the table from the database that contains this character, it works just fine.
my question is: how can I make sure it does show the ë and å and such, and not a ? ?
Your database is probably configured "wrong". When the fields holding your to-be-echoed text are not configured with the proper charset (utf-8), the data you retrieve via php are not either.
So, configure your database properly and insert the data there after that.
Also see How to make MySQL handle UTF-8 properly.
I am reading a varchar(500) column from a SQL Server 2008 R2 database to import into Redshift via a fixedwidth text file.
To pull down the record into a fixed width file, I started out by using a StringBuilder to write out a block of text at a time. I was using AppendFormat and the alignment specifier to align the different records. At certain points, once every 400k lines, I would write the contents of StringBuilder into a StreamWriter to write to disk.
I noticed that there was an issue with text when I tried loading the files into Redshift, the upload into Reshift failed due to extra columns, (there were more columns than my fixed width specification accommodated for).
When I tested the StringBuilder against a regular string, the widths match what I intended them to match, 500 characters.
The discrepancy came when I tried writing my records to disk. I kept getting the same issue when I wrote the aforementioned database column to disk using WriteLineformat the StreamWriter object.
The collation on the database is SQL_Latin1_General_CP1_CI_AS. I understand that strings from the database get converted the database collation to UTF-16. I think there is no problem there, as stated from the test I performed above. I think the issue I'm having is from taking the string in UTF-16 form and writing them to disk using StreamWriter.
I can expect any type of character from the database field, except for a newline or carriage return. I'm pretty confident that white space is trimmed before being pushed into the database column using a combination of the TSQL functions Ltrim and Rtrim.
Edit: Following is the code I use in Powershell
$dw = new-object System.Data.SqlClient.SqlConnection("<connection string details>")
$dw.open()
$reader = (new-object System.Data.SqlClient.Sqlcommand("select email from emails",$dw)).ExecuteReader()
$writer = new-object system.IO.StreamWriter("C:\Emails.txt",[System.Text.Encoding]::UTF8)
while($reader.read())
{
$writer.writelineformat("{0,-500}",$reader["email"])
}
$writer.close()
$reader.close()
Obviously I'm not going to give you the details of my connection string or my table naming convention.
Edit: I'm including the AWS Redshift article that explains that data can only be imported into Redshift using UTF-8 encoding.
http://docs.aws.amazon.com/redshift/latest/dg/t_preparing-input-data.html
Edit: I was able to get a sample of the outputted file through
get-content -encoding utf8
The content inside of the file is definitely UTF-8 proper. All of the line endings within. It's seems like my main issue is with Redshift taking multi-byte characters for fixed width files.
I suspect that the issue is caused by the fact that StreamWriter by default uses UTF-8 so in some instances you will get double byte characters as utf-8 is variable width.
Try using unicode, which will match your database encoding, StreamWriter has an overload which supports encoding.
Just so that anyone seeing this understands. My problem is really with redshift. One thing I have noticed is that the service seems to have processing issues with fixedwidth files. This seems to be specific to Amazon, since the underlying system that runs Redshift is ParAccel. I had issues in the past with Fixedwidth files. I have been able to confirm that there is an issue with Redshift accepting multi-byte characters within fixedwidth version of the S3 Copy command.
I have problem with encoding characters from database. I am using Postgres with win1250 encoding, but whatever I put in core.php (right now I have this line of code):
Configure::write('App.encoding', 'iso-8859-1');
sometimes it give me some strange letters from database, for example È indstead of Č. Is there anything that I can do to get correct encoding.
NOTE: I can't edit or change anything to database.
I think all you need to do is declaring the right encoding option in your database connection configuration, as described at http://book.cakephp.org/2.0/en/development/configuration.html#database-configuration (scroll a bit).
Look at this particular paragraph:
encoding
Indicates the character set to use when sending SQL statements to the server. This defaults to the database’s default encoding for all databases other than DB2. If you wish to use UTF-8 encoding with mysql/mysqli connections you must use ‘utf8’ without the hyphen.
I had the same issue (with French and Spanish names) in a previous project and I only had to add the following to my $default connection, in the app/Config/database.php configuration file:
'encoding' => 'utf8'
Maybe you need the utf8 connection or the iso-8859-1 you mentionned.
win1250 encoding is similar to iso-8859-2 (see http://en.wikipedia.org/wiki/Windows-1250), so you might want to try that instead of iso-8859-1.
Alright
so we had a problem recently
In reporting services some of the String Columns were appearing as gibberish Chinese characters.
On further investigation we found it is the hyphen. Well that's what we though first. On further investigation we found it a dash (or en dash) . Basically the reason this has happened is people copy pasting values into this column from word which converts hyphens into dashes automatically.
But if you look at the database they both look the same. Though on the application side you can see the difference.
How do I replace the dash with a normal hyphen.
If you copy the value in put it in SQL server. A hyphen is gray while a dash is black
but they both look exactly the same (i.e not bigger or smaller). Problem is I can't write a REPLACE script then (they are the freakin same)
REPLACE ('-' with '-')
is there a way special characters like the dash can be identified in SQL server?
SQL Server v 2005
You can use NCHAR(8211) for the en dash, or NCHAR(8212) for em dash.