String Encoding Issue - sql-server

I am having a very weird issue in the way strings get stored in my database, and as a result, I am getting these "unterminated string literal" errors in Javascript.
Here's an overview of what I am doing:
Platform: C#/ASP.NET MVC 1.0, SQL Server 2005, SparkViewEngine, YUI 2
In my view, I serialize an object into a JSON data structure using Json.NET from NewtonSoft.
<script type="text/javascript">
// <![CDATA[
var data = YAHOO.lang.JSON.parse("${Newtonsoft.Json.JsonConvert.Serialize(Model)}");
....
</script>
Normally, this works, but I noticed that one of the fields I pull from the database contains the following data, which causes the string to not be formed properly.
The database field is an NVARCHAR(2000).
For some of the entries, I get these weird characters in the string when I copy and paste from the database to notepad.
Compile ?Analysis ?& ?Recommendations? deck
In Firebug, it shows as a bunch of line breaks:
Analysis & Recommendations deck","StartDate":"1/19/10","FinishDate":"1/26/10","Duration":6.0,"
Compile 
Analysis 
& 
Recommendations
 deck
UPDATE
After talking to the user, I discovered that they were using the copying and pasting from a word document onto the HTML form.
The form itself uses the YUI Connection Manager to make an asynchronous POST call (AJAX) to save the form values.
The database saved the form field value along with any encoding associated with it.
It seems like there are characters in Word that are printable, but not in ASCII. Is there any way to detect this and encode correctly?

This appears to be a case of you not handling the quote characters properly. You have several ways to go. The way I use for comments for our web site is to make sure all quotes are encoded, both single and double, so that both your SQL, ASPX & Javascript won't hiccup.
Our method is for customer comment fields so we're overboard on the conversions.
Function SafeComment(ByVal strInput)
' Renders Any Comment Codes Harmless And Leaves Them HTML readable In An eMail Or Web Page
' Try: SafeComment("`~!##$%^&*()_+=-{}][|\'"";:<>?/.,")
SafeComment = ""
If Len(strInput) = 0 Then Exit Function
SafeComment = Replace(Replace(Replace( _
Replace(Replace(Replace( _
Replace(Replace(Replace( _
Replace(Replace(Replace( _
Server.HtmlEncode(Trim(strInput)), _
"-", "-"), ":", ":"), "|", "|"), _
"`", "`"), "(", "("), ")", ")"), _
"%", "%"), "^", "^"), """", """), _
"/", "/"), "*", "*"), "'", "'")
End Function
It's not pretty, but it does the job.

Related

Storing unicode like ✅ from form to db

I cannot get unicode characters from form to db
The webdesigner/seo-guy wants to use an unicode character ✅ (9989, which shows up as a whute checkmark in a green box.
The data in the CMS is entered through a form, both a
If I paste the ✅ directly into database, using Mssql SSMS, character is seem in the field, and unicode(field) is 9989.
The asp can retrieve the ✅ and put it as value in the html form.
I post the form, the page shows the form sends the correct code, ✅. the page then post the contents and retrieves it again, but the value stored is not ✅ but "?", char 63.
So DB->form is ok, form->DB is broken.
My page starts with
<% Response.Charset="UTF-8"%>
<% Response.codepage="65001" %>
<form method='post' accept-charset="utf-8">
The field in the db is nvarchar, SQL_Latin1_General_CP1_CI_AI though I tried Danish_Norvegian too, no difference.
edit
I have the insert wrapped in a function, but unwrapped it's like this:
Set cmd = Server.CreateObject("ADODB.Command")
Set cmd.ActiveConnection = connection
cmd.CommandType = 1 ' adCmdText
cmd.CommandText = "update t_kat set meta_title=?,meta_description=? where id=?"
cmd.Parameters.Append cmd.CreateParameter("",adVarChar,1,255, request.form("meta_title"))
cmd.Parameters.Append cmd.CreateParameter("",adVarChar,1,255, request.form("meta_description"))
cmd.Parameters.Append cmd.CreateParameter("",adInteger,1,255, request.form("id")
cmd.Execute
EDIT 2
It works if I say server.htmlencode(request.form("meta_description")) but then ✅æøåÆØÅ gets stored in the db as ✅æøåÆØÅ which I'd rather avoid.
In worst case, id accept the "wierd" characters as &#9989 but I prefer the other characters as-is. But I don't think there is an option to select which characters are replaced.
I could do a replace of each "weird" charactes to %#nnnnformat (I don't think he uses that many) in the CMS frontend before storing, but I'd rather not go there either.
SOLVED!
The parameter should be the right type:
cmd.Parameters.Append cmd.CreateParameter("",adVarWChar,1,255, request.form("meta_title")
adVarWChar instead of adVarChar
I think only quotes are in need of being replaced when using the data in a input text form
<input type='text' name="meta_title" value="<%=replace("""",""",res("meta_title"))%>">
This is to prevent invalid HTML like
<input type="text" value="The 11'8" bridge">
Turning it into <input type="text" value="The 11'8" bridge">
If used as text, the < needs escaping;
<B>one<two</B>

Getting Run-Time error '1004' Is my formula perhaps too long or can I fix the coding?

So I wrote up an Array formula which works fine on my excel, but when converting it to VBA I'm getting an error: Unable to set the FormulaArray property of the Range class. I'm not sure why I'm getting this issue. Please see the code here:
Selection.FormulaArray = _
"=IF(IF(AND(OR(RC[-14]<>R[-1]C[-14],RC[-15]<>R[-1]C[-15]),Pay_Periods!R2C10<>0),MAX(IF(Pay_Periods!R2C3:R250C3>=Pay_Periods!R1C10+(14*Pay_Periods!R3C10),IF(Pay_Periods!R2C2:R250C2<=Pay_Periods!R1C10+(14*Pay_Periods!R3C10),Pay_Periods!R2C3:R250C3))),MAX(IF((Pay_Periods!R2C3:R250C3>=Sheet1!RC[-10])*(IF(AND(Sheet1!RC[-14]=Sheet1!R[-1]C[-14],Sheet1!R[-1]C[-10]<Sheet1!RC[" & _
"-10]+14),Pay_Periods!R2C3:R250C3<(RC[-10]+14),IF(AND(RC[-14]=R[-1]C[-14],RC[-15]=R[-1]C[-15]),Pay_Periods!R2C3:R250C3<R[-1]C[-10],1))),Pay_Periods!R2C3:R250C3,"""")))=0,"""",IF(AND(OR(RC[-14]<>R[-1]C[-14],RC[-15]<>R[-1]C[-15]),Pay_Periods!R2C10<>0),MAX(IF(Pay_Periods!R2C3:R250C3>=Pay_Periods!R1C10+(14*Pay_Periods!R3C10),IF(Pay_Periods!R2C2:R250C2<=Pay_Periods!R1C10+" & _
"(14*Pay_Periods!R3C10),Pay_Periods!R2C3:R250C3))),MAX(IF((Pay_Periods!R2C3:R250C3>=Sheet1!RC[-10])*(IF(AND(Sheet1!RC[-14]=Sheet1!R[-1]C[-14],Sheet1!RC[-15]=Sheet1!R[-1]C[-15],Sheet1!R[-1]C[-10]<Sheet1!RC[-10]+14),Pay_Periods!R2C3:R250C3<(RC[-10]+14),IF(AND(RC[-14]=R[-1]C[-14],RC[-15]=R[-1]C[-15]),Pay_Periods!R2C3:R250C3<R[-1]C[-10],1))),Pay_Periods!R2C3:R250C3,"""")" & _
")))" & _
""
By the way I copied it exactly how the macro itself recorded it so you may notice some spaces. I'm thinking it could just be too long, but it seems weird that writing it in the cell gets it fine, but not in the vba form. If anyone can help it would be much appreciated! I'm attempting to have this formula self insert by code and copy paste values so the excel doesn't have to keep loading in information.
Here is how the formula looks normally:
=IF(IF(AND(OR(C2<>C1,B2<>B1),Pay_Periods!$J$2<>0),MAX(IF(Pay_Periods!$C$2:$C$250>=Pay_Periods!$J$1+(14*Pay_Periods!$J$3),IF(Pay_Periods!$B$2:$B$250<=Pay_Periods!$J$1+(14*Pay_Periods!$J$3),Pay_Periods!$C$2:$C$250))),MAX(IF((Pay_Periods!$C$2:$C$250>=Sheet1!G2)*(IF(AND(Sheet1!C2=Sheet1!C1,Sheet1!G1<Sheet1!G2+14),Pay_Periods!$C$2:$C$250<(G2+14),IF(AND(C2=C1,B2=B1),Pay_Periods!$C$2:$C$250<G1,1))),Pay_Periods!$C$2:$C$250,"")))=0,"",IF(AND(OR(C2<>C1,B2<>B1),Pay_Periods!$J$2<>0),MAX(IF(Pay_Periods!$C$2:$C$250>=Pay_Periods!$J$1+(14*Pay_Periods!$J$3),IF(Pay_Periods!$B$2:$B$250<=Pay_Periods!$J$1+(14*Pay_Periods!$J$3),Pay_Periods!$C$2:$C$250))),MAX(IF((Pay_Periods!$C$2:$C$250>=Sheet1!G2)*(IF(AND(Sheet1!C2=Sheet1!C1,Sheet1!B2=Sheet1!B1,Sheet1!G1<Sheet1!G2+14),Pay_Periods!$C$2:$C$250<(G2+14),IF(AND(C2=C1,B2=B1),Pay_Periods!$C$2:$C$250<G1,1))),Pay_Periods!$C$2:$C$250,""))))
Then ctrl+shift+enter naturally
I went ahead and built a custom solution just for you. Here, you'll see I broke up your formula into smaller parts and substitute them back together (just like the good 'ol days back in algebra class).
Also note, that I don't use the Selection object as you do in your question. I recommend working with Ranges directly rather than using .Select, Selection. or .Activate and so on. So In my example below I assume the range you have "Selected" is A1 on the first worksheet.
strFormula = "=IF(IF(AND(OR(C2<>C1,B2<>B1),Pay_Periods!$J$2<>0),MAX(W_W),MAX(X_X))=0,""""," & _
"IF(AND(OR(C2<>C1,B2<>B1),Pay_Periods!$J$2<>0),MAX(Y_Y),MAX(Z_Z)))"
strFormulaW_W = "IF(Pay_Periods!$C$2:$C$250>=Pay_Periods!$J$1+(14*Pay_Periods!$J$3)," & _
"IF(Pay_Periods!$B$2:$B$250<=Pay_Periods!$J$1+(14*Pay_Periods!$J$3),Pay_Periods!$C$2:$C$250))"
strFormulaX_X = "IF((Pay_Periods!$C$2:$C$250>=Sheet1!G2)*(IF(AND(Sheet1!C2=Sheet1!C1,Sheet1!G1<Sheet1!G2+14)," & _
"Pay_Periods!$C$2:$C$250<(G2+14),IF(AND(C2=C1,B2=B1),Pay_Periods!$C$2:$C$250<G1,1))),Pay_Periods!$C$2:$C$250,"""")"
strFormulaY_Y = "IF(Pay_Periods!$C$2:$C$250>=Pay_Periods!$J$1+(14*Pay_Periods!$J$3),IF(Pay_Periods!$B$2:$B$250<=" & _
"Pay_Periods!$J$1+(14*Pay_Periods!$J$3),Pay_Periods!$C$2:$C$250))"
strFormulaZ_Z = "IF((Pay_Periods!$C$2:$C$250>=Sheet1!G2)*(IF(AND(Sheet1!C2=Sheet1!C1,Sheet1!B2=Sheet1!B1," & _
"Sheet1!G1<Sheet1!G2+14),Pay_Periods!$C$2:$C$250<(G2+14),IF(AND(C2=C1,B2=B1),Pay_Periods!$C$2:$C$250<G1,1))),Pay_Periods!$C$2:$C$250,"""")"
With ThisWorkbook.Worksheets(1).Range("A1")
.FormulaArray = strFormula
.Replace "W_W", strFormulaW_W
.Replace "X_X", strFormulaX_X
.Replace "Y_Y", strFormulaY_Y
.Replace "Z_Z", strFormulaZ_Z
End With

Golang CSV error bare " in non-quoted-field

I haven't had trouble parsing csv files for my GAE golang app until this week (I updated to appengine 1.9.23 last week). Now, regardless of file content I am getting this error:
2015/07/09 15:25:34 http: panic serving 127.0.0.1:50352: line 1, column 22: bare " in non-quoted-field
Even when the file content doesn't contain any " characters at all the error occurs.
Anyone know why my files can no longer be parsed? Something changed or I'm doing something super-stupid.
PS using urlfetch to obtain the csv file
This happens when we have on CSV file de " (double quotes) value.
To avoid this error we should use LazyQuotes Parameter like that:
csvFile, _ := os.Open("file.csv")
reader := csv.NewReader(bufio.NewReader(csvFile))
reader.Comma = ';'
reader.LazyQuotes = true
After much ado I determined that the hosting company had updated DotDefender which introduced a rule to block .csv/.tsv arg
If the csv decode library follows RFC-4180
If double-quotes are used to enclose fields, then a double-quote
appearing inside a field must be escaped by preceding it with
another double quote.
For example:
"aaa","b""bb","ccc"

Some files have 'XMLNS' listed, others just 'XSI'. How do I set an alias for the xpath query that will work with either?

Some files I need to process have this (called the "haves"):
<ApolloDataSet xmlns="http://irisoft.com/ApolloDataSet1.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
Other files in the same group have this (called the "have-nots"):
<ApolloDataSet xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
I can set the default namespace for xpath with the below:
.setProperty "SelectionNamespaces", "xmlns:a='http://irisoft.com/ApolloDataSet1.xsd'"
That works for the haves, like a:/Element, but not the have-nots, since the xpath doesn't have the a: alias.
I tried removing the xmlns attribute before processing, in the hopes that I could just use an unaliased path for both, like /Element, but that only worked for the have-nots (the haves just returned nothing).
So is there a way to process both using the same alias, or no alias? I'm trying to either use the same alias for every file, regardless of "xmlns" being listed, or use no alias for either.
Well I resigned to using the workaround I was trying to avoid. But here is an option for this situation until I find a better one. (And forgot to mention - using VBA for this.)
After the domDocument is loaded, I use the getAttribute method to check if the "xmlns" attribute value is null. If it is I form the xpath string with no alias by means of a vbnullstring, which means the xpath query works with the have-not files (i.e. those files with no "xmlns" listed). If it is not null, I use the "xmlns" value to build a string for the setProperty method setting the "SelectionNamespaces" secondary property.
If IsNull(xPOG.DocumentElement.getAttribute("xmlns")) Then
strAlias = vbNullString
Else
strAlias = "a:"
xPOG.setProperty "SelectionNamespaces", _
"xmlns:" & Mid(strAlias, 1, 1) & "='" & xPOG.DocumentElement.getAttribute("xmlns") & "'"
End If
And the resulting processing takes on a rather ugly form of string building that I hoped to avoid.
It sets an IXMLDOMNode to the nodes returned from a selectnodes method using the xPath query with the strAlias variable.
Set xProducts = xPOG.SelectNodes("/" & strAlias & "ApolloDataSet/" _
& strAlias & "Products/" _
& strAlias & "Product/" _
& strAlias & "PR_UPC")
It isn't that it is bad, it just seems to lack elegance & remind me of how little I know sometimes. Which leads me to believe I'm missing some concept that could correct this. But it works, so I'll follow up if I find a better. I did find a kb article that touched on some of the concepts if it helps anyone. I'm using 6.0 though, so good conceptual explanation, just not exactly what I needed.
PRB: MSXML 4.0 Sets the XML Namespace Attribute to an Empty Value for Child Nodes

Crystal Report issue.Net 4.0

There is a definite syntax of passing value to crystal-report Formula Field from .Net 4.0- Code
behind
E.g. (C# Code):
Following will NOT work
_rpt.DataDefinition.FormulaFields["fClient"].Text = Mr.Gates;
Following will work-
_rpt.DataDefinition.FormulaFields["fClient"].Text = "\"Mr.Gates\"";
I haven't understand the prominence and way of using following ..
"\" \""
What if "Mr. Gates:" is in some variable, I want to pass!?
plz reply with some code...
Thanks
\" sends a literal " through to the variable. For literal string values in Formulae to be seen as literal strings they must be quoted hence sending the quoted " through is necessary.

Resources