I have model I created on the fly for peewee. Something like this:
class TestTable(PeeweeBaseModel):
whencreated_dt = DateTimeField(null=True)
whenchanged = CharField(max_length=50, null=True)
I load data from a text file to a table using peewee, the column "whenchanged" contains all dates in a format of '%Y-%m-%d %H:%M:%S' as varchar column. Now I want to convert the text field "whenchanged" into a datetime format in "whencreated_dt".
I tried several things... I ended up with this:
# Initialize table to TestTable
to_execute = "table.update({table.%s : datetime.strptime(table.%s, '%%Y-%%m-%%d %%H:%%M:%%S')}).execute()" % ('whencreated_dt', 'whencreated')
which fails with a "TypeError: strptime() argument 1 must be str, not CharField": I'm trying to convert "whencreated" to datetime and then assign it to "whencreated_dt".
I tried a variation... following e.g. works without a hitch:
# Initialize table to TestTable
to_execute = "table.update({table.%s : datetime.now()}).execute()" % (self.name)
exec(to_execute)
But this is of course just the current datetime, and not another field.
Anyone knows a solution to this?
Edit... I did find a workaround eventually... but I'm still looking for a better solution... The workaround:
all_objects = table.select()
for o in all_objects:
datetime_str = getattr( o, 'whencreated' )
setattr(o, 'whencreated_dt', datetime.strptime(datetime_str, '%Y-%m-%d %H:%M:%S'))
o.save()
Loop over all rows in the table, get the "whencreated". Convert "whencreated" to a datetime, put it in "whencreated_dt", and save each row.
Regards,
Sven
Your example:
to_execute = "table.update({table.%s : datetime.strptime(table.%s, '%%Y-%%m-%%d %%H:%%M:%%S')}).execute()" % ('whencreated_dt', 'whencreated')
Will not work. Why? Because datetime.strptime is a Python function and operates in Python. An UPDATE query works in database-land. How the hell is the database going to magically pass row values into "datetime.strptime"? How would the db even know how to call such a function?
Instead you need to use a SQL function -- a function that is executed by the database. For example, Postgres:
TestTable.update(whencreated_dt=whenchanged.cast('timestamp')).execute()
This is the equivalent SQL:
UPDATE test_table SET whencreated_dt = CAST(whenchanged AS timestamp);
That should populate the column for you using the correct data type. For other databases, consult their manuals. Note that SQLite does not have a dedicated date/time data type, and the datetime functionality uses strings in the Y-m-d H:M:S format.
I have a groovy program with code as below:
String[] conceptArray = conceptList.toArray()
I am trying to insert conceptArray into Oracle table like below
sql.execute("insert into OFFICE.PERSON(CONCEPT) values (?)",[conceptArray]);
But the above code is not inserting values into the table Person. Please help me to correct the above code.
Use the groovy.sql.Sql class:
http://docs.groovy-lang.org/latest/html/api/groovy/sql/Sql.html
def sql=new groovy.sql.Sql(...)
String insert="insert into OFFICE.PERSON values (${conceptList.collect{'?'}.join(','})"
sql.executeUpdate(insert,conceptList)
The SQL prepared statement (which is created with the GString and ? syntax) is expecting a string to insert where the ? is, but you are passing it a List. You will need to turn the list of values into a String, which you could do this way:
String conceptsString = conceptList.join(',')
sql.execute("insert into OFFICE.PERSON(CONCEPT) values (?)", [conceptsString])
I have many json arrays stored in a table (jt) that looks like this:
[{"ts":1403781896,"id":14,"log":"show"},{"ts":1403781896,"id":14,"log":"start"}]
[{"ts":1403781911,"id":14,"log":"press"},{"ts":1403781911,"id":14,"log":"press"}]
Each array is a record.
I would like to parse this table in order to get a new table (logs) with 3 fields: ts, id, log.
I tried to use the get_json_object method, but it seems that method is not compatible with json arrays because I only get null values.
This is the code I have tested:
CREATE TABLE logs AS
SELECT get_json_object(jt.value, '$.ts') AS ts,
get_json_object(jt.value, '$.id') AS id,
get_json_object(jt.value, '$.log') AS log
FROM jt;
I tried to use other functions but they seem really complicated.
Thank you! :)
Update!
I solved my issue by performing a regexp:
CREATE TABLE jt_reg AS
select regexp_replace(regexp_replace(value,'\\}\\,\\{','\\}\\\n\\{'),'\\[|\\]','') as valuereg from jt;
CREATE TABLE logs AS
SELECT get_json_object(jt_reg.valuereg, '$.ts') AS ts,
get_json_object(jt_reg.valuereg, '$.id') AS id,
get_json_object(jt_reg.valuereg, '$.log') AS log
FROM ams_json_reg;
I just ran into this problem, with the JSON array stored as a string in the hive table.
The solution is a bit hacky and ugly, but it works and doesn't require serdes or external UDFs
SELECT
get_json_object(single_json_table.single_json, '$.ts') AS ts,
get_json_object(single_json_table.single_json, '$.id') AS id,
get_json_object(single_json_table.single_json, '$.log') AS log
FROM ( SELECT explode (
split(regexp_replace(substr(json_array_col, 2, length(json_array_col)-2),
'"}","', '"}",,,,"'), ',,,,')
) FROM src_table) single_json_table;
I broke the lines up so that it would be a little easier to read.
I'm using substr() to strip the first and last characters, removing [ and ] . I'm then using regex_replace to match the separator between records in the json array and adding or changing the separator to be something unique that can then be used easily with split() to turn the string into a hive array of json objects which can then be used with explode() as described in the previous solution.
Note, the separator regex used here ( "}"," ) wouldn't work with the original data set...the regex would have to be ( "},\{" ) and the replacement would then need to be "},,,,{" eg..
split(regexp_replace(substr(json_array_col, 2, length(json_array_col)-2),
'"},\\{"', '"},,,,{"'), ',,,,')
Use explode() function
hive (default)> CREATE TABLE logs AS
> SELECT get_json_object(single_json_table.single_json, '$.ts') AS ts,
> get_json_object(single_json_table.single_json, '$.id') AS id,
> get_json_object(single_json_table.single_json, '$.log') AS log
> FROM
> (SELECT explode(json_array_col) as single_json FROM jt) single_json_table ;
Automatically selecting local only mode for query
Total MapReduce jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
hive (default)> select * from logs;
OK
ts id log
1403781896 14 show
1403781896 14 start
1403781911 14 press
1403781911 14 press
Time taken: 0.118 seconds, Fetched: 4 row(s)
hive (default)>
where json_array_col is column in jt which holds your array of jsons.
hive (default)> select json_array_col from jt;
json_array_col
["{"ts":1403781896,"id":14,"log":"show"}","{"ts":1403781896,"id":14,"log":"start"}"]
["{"ts":1403781911,"id":14,"log":"press"}","{"ts":1403781911,"id":14,"log":"press"}"]
because get_json_object doesn't support json array string, so you can concat to a json object, like this:
SELECT
get_json_object(concat(concat('{"root":', jt.value), '}'), '$.root')
FROM jt;
Hello I have the following xml structure within a database table column :
DECLARE #Response XML =
'<star:ShowInfo xmlns="http://www.starstandard.org/STAR/5"
xmlns:ns2="http://www.openapplications.org/oagis/9"
xmlns:star="http://www.starstandard.org/STAR/5"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" releaseID="5.1.5"
xsi:noNamespaceSchemaLocation="">
<ShowDataArea>
<ServiceInfo>
<SVPlanInfo>
<AKStatus>
<Code>Error</Code>
<STText xsi:type="ns2:TextType">E12143 - Please fetch me from this xml </STText>
</AKStatus>
</SVPlanInfo>
</ServiceInfo>
</ShowDataArea></star:ShowInfo>'
In the above xml I need to fetch the STText value which is
E12143 - Please fetch me from this xml . Can anyone point me on how I can do it ?
I tried the following but it doesnt seem to work :
;WITH XMLNAMESPACES ('http://www.w3.org/2001/XMLSchema' as xsd,
'http://www.w3.org/2001/XMLSchema-instance' as xsi)
SELECT #Response.value('(/xsd:Response)[1]','nvarchar(500)') as ExceptionMessage
What a pain.
remove:
xmlns="http://www.starstandard.org/STAR/5"
It is not a sql issue, but rather namespace-getting-confused issue.
Its heplful totake SQL out the equation sometimes, by testing some place like
http://xpath.online-toolz.com/tools/xpath-editor.php.
I have a column that is NVarChar(MAX), it contains text like this: ;0,4,6
These are the Flag values from another table.
A. I set the EditValueFormat propertie of the component to cvfIndices
B. I put the table values from Table a to a TcxDBCheckComboBox component using code like this:
Query.Active := True;
while not Query.Eof do begin
cxDBCheckComboBox1.Properties.Items.AddCheckItem(QueryCaptionField.AsString);
Query.Next;
end;
Query.Active := False;
C. I assigned a datasource to point to that column I want to hold my value when I select the values in the program the text looks like this in the database: ;0,4,6 how should I query it to find what is selected from SQL Server?
Found the answer while using cvfIndices the value saved in the database represents the index of the value in the combobox so you can query it like this:
SELECT * FROM TechCardsData
LEFT JOIN TechCardsOperations
on
CHARINDEX(CAST(TechCardsOperations.ID-1 as nvarchar(max)),TechCardsData.OperationsFlags) <> 0
and everything works :) hope this helps someone :)