Problem:
When I decode and reconstitute my base64 encoded XML document, I get null values between each character. I think I need to convert to NCHAR, but it's not working as expected.
Block : PgA8AC8AYwBsAGkAbgBpAGMAYQBsAF8AcwB0AHUAZAB5AD4A
Decode : 3E003C002F0063006C0069006E006900630061006C005F00730074007500640079003E00
Varchar2: > < / c l i n i c a l _ s t u d y
Raw2NHex: 3E003C002F0063006C0069006E006900630061006C005F00730074007500640079003E00
(Note: The blank characters on the Varchar2 output are actually nulls.
Background:
I am having difficulty reading an XML data column from a base64 encoded element in an XML document that I build in SQL Server.
Basically, we are moving an XML document from one system to another, and had to use Base64 because Unicode < characters that were not < tag characters were being translated as <. Namely the text said < 3 months, using the unicode less than sign.
So, we translated the xml document into Base64 which worked great.
Sample XML Document
My Oracle decode function is this
v_clob := '';
v_offset := 1;
FOR i IN 1 .. ceil(dbms_lob.getlength(p_clob_in) / v_buffer_size) LOOP
--
-- Substr is 1-relative.
--
dbms_lob.read(p_clob_in, v_buffer_size, v_offset, v_buffer_base64);
--
IF v_buffer_base64 IS NULL THEN
EXIT;
END IF;
DBMS_OUTPUT.PUT_LINE('Block : ' || SUBSTR(v_buffer_base64, 1, 80));
--
-- Decode the Base64 into a RAW string
-- ... This works as expected
--
v_buffer_decode := utl_encode.base64_decode(utl_raw.cast_to_raw(v_buffer_base64));
DBMS_OUTPUT.PUT_LINE('Decode : ' || v_buffer_decode);
--
-- Cast the RAW output into VARCHAR2
-- ... This results in null characters between each character
--
v_buffer_varchar2 := utl_raw.cast_to_varchar2(v_buffer_decode);
DBMS_OUTPUT.PUT_LINE('Varchar2: ' || v_buffer_varchar2);
--
-- Convert the decoded RAW string into NVARCHAR
-- ... This doesn't actually do anything, it just puts out the same RAW characters.
-- ... I get the same result as I do with RAWTONHEX.
--
SELECT CAST(v_buffer_decode as NVARCHAR2(1024))
INTO v_buffer_nvarchar2
FROM dual;
DBMS_OUTPUT.PUT_LINE('CastNVC : ' || v_buffer_nvarchar2);
v_clob := v_clob || v_buffer_nhex;
--
v_offset := v_offset + v_buffer_size;
--
END LOOP;
What I need is something that can take the 2-byte NCHAR and represent it as the proper single NCHAR value in the NVARCHAR2 variable.
After struggling with this and reviewing the debug statements, I came to the realization that Microsoft and Oracle treat the high order byte of wide character sets differently.
In Microsoft, the binary is rendered as 3E00. However, if you use
select '*' || utl_raw.cast_to_nvarchar2('3e00') || '*' from dual
You get *γΈ€*. If, however, you do
select '*' || utl_raw.cast_to_nvarchar2('003e') || '*' from dual
You get *>*. Which is the proper conversion.
So, what I had to do was to extract characters from the decoded RAW string and flip the low-order and high-order bytes so that 3E00 became 003E. I also, in the end, had to skip the FFFE base64 prefix from the binary data since Oracle didn't throw it away.
Here is the final code.
FUNCTION decode_base64(p_clob_in IN CLOB) RETURN CLOB IS
--
v_clob clob;
v_result clob;
v_offset number;
v_buffer_size binary_integer := 48;
v_buffer_base64 varchar2(1024);
v_buffer_decode RAW(1024);
v_buffer_varchar2 VARCHAR2(1024);
v_buffer_nhex nvarchar2(1024);
v_buffer_nvarchar2 nvarchar2(1024);
v_buffer_nchar NVARCHAR2(4);
v_buffer_convert RAW(4);
--
BEGIN
--
IF p_clob_in IS NULL THEN
RETURN NULL;
END IF;
--
dbms_lob.createtemporary(v_clob, true);
--
v_clob := '';
v_offset := 1;
FOR i IN 1 .. ceil(dbms_lob.getlength(p_clob_in) / v_buffer_size) LOOP
--
-- Substr is 1-relative.
--
dbms_lob.read(p_clob_in, v_buffer_size, v_offset, v_buffer_base64);
--
IF v_buffer_base64 IS NULL THEN
EXIT;
END IF;
--
-- Decode the Base64 into a RAW string
--
v_buffer_decode := utl_encode.base64_decode(utl_raw.cast_to_raw(v_buffer_base64));
--
-- Convert the decoded text into NVARCHAR2 characters
-- Note: Microsoft and Oracle disagree on high byte first or second
-- Take out each character (4 characters) and flip the 3rd and 4th hex character to the front
-- Thus, 3E00 becomes 003E
-- Note: We have to ignore Microsoft's Base64 prefix code of FFFE. (FEFF when extracted :) ).
--
v_buffer_nvarchar2 := '';
FOR J IN 1 .. ceil(LENGTH(v_buffer_decode) / 4) LOOP
--
v_buffer_convert := SUBSTR(v_buffer_decode, (J - 1) * 4 + 3, 2) || SUBSTR(v_buffer_decode, (J - 1) * 4 + 1, 2);
IF v_buffer_convert <> 'FEFF' THEN
v_buffer_nchar := utl_raw.cast_to_nvarchar2(v_buffer_convert);
v_buffer_nvarchar2 := v_buffer_nvarchar2 || v_buffer_nchar;
END IF;
--
END LOOP;
v_clob := v_clob || v_buffer_nvarchar2;
--
v_offset := v_offset + v_buffer_size;
--
END LOOP;
--
v_result := v_clob;
dbms_lob.freetemporary(v_clob);
RETURN v_result;
--
END decode_base64;
That was the fix.
Related
I have a function that remove last character from a varchar2, but I needed to convert it to char array first. Now I cant find anything to convert it back to varchar2.
My function:
DECLARE
TYPE CHAR_ARRAY IS TABLE OF CHAR(1) INDEX BY PLS_INTEGER;
NAME VARCHAR2(100) := '&vname';
NAME_CHAR CHAR_ARRAY;
BEGIN
FOR X IN 1..LENGTH(NAME) LOOP
IF((X = LENGTH(NAME))) THEN
NAME_CHAR(X) := '';
ELSE
NAME_CHAR(X) := SUBSTR(NAME, X , 1);
END IF;
END LOOP;
-- Now I need to convert it back to varchar2
END;
How about:
name := '';
for x in 1..name_char.count loop
name := name || name_char(x);
end loop;
Though why you would do any of this eludes me! If I wanted to remove the last character from a string I would do this:
name := substr (name, 1, length(name)-1);
How to split a comma delimited string of numbers into DBMS_UTILITY.NUMBER_ARRAY?
SET serveroutput on;
declare
l_tablen BINARY_INTEGER;
l_tab DBMS_UTILITY.NUMBER_ARRAY;
begin
DBMS_UTILITY.comma_to_table ('1,2,3', l_tablen, l_tab);
FOR i IN 1 .. l_tablen LOOP
DBMS_OUTPUT.put_line(i || ' : ' || l_tab(i));
END LOOP;
end;
This of course gives us
PLS-00306: wrong number or types of arguments in call to 'COMMA_TO_TABLE'
The type of your variable l_tab should be DBMS_UTILITY.UNCL_ARRAY and not DBMS_UTILITY.NUMBER_ARRAY. Cheers
I want to execute some sql statements stored in a clob in the database.
I want to make use of dbms_sql.parse with a clob as input parameter.
The code I tried as a testcase on a 11.2 Oracle database:
Making the table for the inserts:
create table table1 (t1 number(8), t2 varchar2(1), t3 varchar2(1));
The statement that fails:
DECLARE
cursor makeclob is
select 'insert into table1 (t1,t2,t3) values ('||rownum||', ''X'',''I'');' stat
from dual
connect by level < 10000;
testcl clob;
opencu integer;
err integer;
BEGIN
for rec in makeclob loop
testcl := testcl || rec.stat || '\n';
end loop;
testcl := testcl || 'commit;'|| '\n';
opencu := dbms_sql.open_cursor;
dbms_sql.parse(opencu,testcl,dbms_sql.native);
err := dbms_sql.execute(opencu);
dbms_sql.close_cursor(opencu);
END;
This statement failed with the following error:
ORA-00911: invalid character.
ORA-06512: at "SYS.DBMS_SQL", line 1250
ORA-06512: at line 17
00911. 00000 - "invalid character"
*Cause: identifiers may not start with any ASCII character other than
letters and numbers. $#_ are also allowed after the first
character. Identifiers enclosed by doublequotes may contain
any character other than a doublequote. Alternative quotes
(q'#...#') cannot use spaces, tabs, or carriage returns as
delimiters. For all other contexts, consult the SQL Language
Reference Manual.
*Action:
Does anyone know what is wrong with my statement?
You should wrap your parsed statements between
BEGIN
and
END
Also use chr(13) instead of '\n'.
I ajusted your code a little bit, so take a look at this:
DECLARE
cursor makeclob is
select 'insert into table1 (t1,t2,t3) values ('||rownum||', ''X'',''I'');' stat
from dual
connect by level < 10000;
testcl clob;
opencu integer;
err integer;
BEGIN
testcl := 'BEGIN'||chr(13);
for rec in makeclob loop
testcl := testcl || rec.stat ||chr(13);
end loop;
testcl := testcl || 'commit;'||chr(13);
testcl := testcl || 'END;';
opencu := dbms_sql.open_cursor;
dbms_sql.parse(opencu,testcl,dbms_sql.native);
err := dbms_sql.execute(opencu);
dbms_sql.close_cursor(opencu);
END;
This is Oracle 11.2g. In a PL/SQL function, I've got a loop whereby each iteration, I create a string and an integer associated with that string. The function returns the final concatenation of all the generated strings, sorted (depending on a function input parameter), either alphabetically or by the value of the integer. To give an idea, I'm generating something like this:
Iteration String Integer
1 Oslo 40
2 Berlin 74
3 Rome 25
4 Paris 10
If the input parameter says to sort alphabetically, the function output should look like this :
Berlin, Oslo, Paris, Rome
Otherwise, we return the concatenated strings sorted by the value of the associated integer:
Paris, Rome, Oslo, Berlin
What is the most appropriate data structure to achieve this sort? I've looked at collections, associative arrays and even varrays. I've been kind of shocked how difficult this seems to be to achieve in Oracle. I saw this question but it doesn't work in my case, as I need to be able to sort by both index and value: How to sort an associative array in PL/SQL? Is there a more appropriate data structure for this scenario, and how would you sort it?
Thanks!
It is very easy if you use PL/SQL as SQL and not like other languages. It is quite specific and sometimes is very nice exactly because of that.
Sometimes I really hate PL/SQL, but this case is absolutely about love.
See how easy it is:
create type it as object (
iter number,
stringval varchar2(100),
intval integer
);
create type t_it as table of it;
declare
t t_it := new t_it();
tmp1 varchar2(32767);
tmp2 varchar2(32767);
begin
t.extend(4);
t(1) := new it(1,'Oslo',40);
t(2) := new it(2,'Berlin',74);
t(3) := new it(3,'Rome',25);
t(4) := new it(4,'Paris',10);
select listagg(stringval,', ') within group (order by stringval),
listagg(stringval,', ') within group (order by intval)
into tmp1, tmp2
from table(t);
dbms_output.put_line(tmp1);
dbms_output.put_line(tmp2);
end;
/
drop type t_it;
drop type it;
Here you can see the problem that you must create global types, and this is what I hate it for. But they say in Oracle 12 it can be done with locally defined types so I am waiting for it :)
The output is:
Berlin, Oslo, Paris, Rome
Paris, Rome, Oslo, Berlin
EDIT
As far as you do not know the amount of iterations from the beginning the only way is to do extend on each iteration (this is only example of extending):
declare
iterator pls_integer := 1;
begin
/* some type of loop*/ loop
t.extend();
-- one way to assign
t(t.last) := new it(1,'Oslo',40);
-- another way is to use some integer iterator
t(iterator) := new it(1,'Oslo',40);
iterator := iterator + 1;
end loop;
end;
I prefer the second way because it is faster (does not calculate .last on each iteration).
This is an example of pure PL/SQL implementation that is based on the idea associative array (aka map or dictionary in other domains) is an ordered collection that is sorted by a key. That is a powerful feature that I have used multiple times. For input data structure in this example I decided to use a nested table of records (aka a list of records).
In this particular case however I'd probably go for similar implementation than in simon's answer.
create or replace package so36 is
-- input data structures
type rec_t is record (
iter number,
str varchar2(20),
int number
);
type rec_list_t is table of rec_t;
function to_str(p_list in rec_list_t, p_sort in varchar2 default 'S')
return varchar2;
end;
/
show errors
create or replace package body so36 is
function to_str(p_list in rec_list_t, p_sort in varchar2 default 'S')
return varchar2 is
v_sep constant varchar2(2) := ', ';
v_ret varchar2(32767);
begin
if p_sort = 'S' then
-- create associative array (map) v_map where key is rec_t.str
-- this means the records are sorted by rec_t.str
declare
type map_t is table of rec_t index by varchar2(20);
v_map map_t;
v_key varchar2(20);
begin
-- populate the map
for i in p_list.first .. p_list.last loop
v_map(p_list(i).str) := p_list(i);
end loop;
v_key := v_map.first;
-- generate output string
while v_key is not null loop
v_ret := v_ret || v_map(v_key).str || v_sep;
v_key := v_map.next(v_key);
end loop;
end;
elsif p_sort = 'I' then
-- this branch is identical except the associative array's key is
-- rec_t.int and thus the records are sorted by rec_t.int
declare
type map_t is table of rec_t index by pls_integer;
v_map map_t;
v_key pls_integer;
begin
for i in p_list.first .. p_list.last loop
v_map(p_list(i).int) := p_list(i);
end loop;
v_key := v_map.first;
while v_key is not null loop
v_ret := v_ret || v_map(v_key).str || v_sep;
v_key := v_map.next(v_key);
end loop;
end;
end if;
return rtrim(v_ret, v_sep);
end;
end;
/
show errors
declare
v_list so36.rec_list_t := so36.rec_list_t();
v_item so36.rec_t;
begin
v_item.iter := 1;
v_item.str := 'Oslo';
v_item.int := 40;
v_list.extend(1);
v_list(v_list.last) := v_item;
v_item.iter := 2;
v_item.str := 'Berlin';
v_item.int := 74;
v_list.extend(1);
v_list(v_list.last) := v_item;
v_item.iter := 3;
v_item.str := 'Rome';
v_item.int := 25;
v_list.extend(1);
v_list(v_list.last) := v_item;
v_item.iter := 4;
v_item.str := 'Paris';
v_item.int := 10;
v_list.extend(1);
v_list(v_list.last) := v_item;
dbms_output.put_line(so36.to_str(v_list));
dbms_output.put_line(so36.to_str(v_list, 'I'));
end;
/
show errors
This is my procedure.
procedure format_integer_field(Atable: TDataSet);
var i: integer;
begin
if Atable.Active then
if Atable.FieldCount > 0 then
with Atable do
begin
for i:= 0 to FieldCount-1 do
if (Fields[i] is TIntegerField) then
begin
(Fields[i] as TIntegerField).DisplayFormat := '###,###';
(Fields[i] as TIntegerField).EditFormat := '#';
end
else
if (Fields[i] is TFloatField) then
begin
(Fields[i] as TFloatField).DisplayFormat := '###,###.##';
(Fields[i] as TFloatField).EditFormat := '#.##';
end;
end;
end;
This is work fine until a number like "0.9" has been entered and result will be ".9".
How can I have thousand separator and zero before floating point that smaller than "1".
Try (Fields[i] as TFloatField).DisplayFormat := '##0,000.00';
As you did read in documentation at http://docwiki.embarcadero.com/RADStudio/XE3/en/Using_Default_Formatting_for_Numeric,_Date,_and_Time_Fields it says
Default formatting is performed by the following routines:
FormatFloat -- TFloatField, TCurrencyField
And how you did read in the following documentation pages
http://docwiki.embarcadero.com/Libraries/XE3/en/System.SysUtils.FormatFloat
http://docwiki.embarcadero.com/Libraries/XE3/en/Data.DB.TNumericField.DisplayFormat
the documentation quotes
0 -> Digit placeholder. If the value being formatted has a digit in the position where '0' appears in the format string, then
that digit is copied to the output string. Otherwise, a '0' is
stored in that position in the output string.
# -> Digit placeholder. If the value being formatted has a digit in the position where '#' appears in the format string, then
that digit is copied to the output string. Otherwise, nothing is
stored in that position in the output string.
So by using "#" in the formatting pattern you tell Delphi "i do not need any digits (and thousands separators with them) in this place, but you might put them if you want" - and since Delphi does not want to put leading zeros - you don't have any. However, if you really need those digits and the thousands separator with them, you put "0" instead of "#" and that way you tell Delphi "the digits just need to be here, whether you want to put them or not"
The format you need is ###,##0.0#