just wondering, if MySQL can be used in corona or MySQL db's can be opened using sqlite3
or is there any difference If I use a MySQL db or sqlite3 db?
Corona provides its own library for connection to SQLite drivers, whose document can be accessed here.
local sqlite3 = require "sqlite3"
--Open data.db. If the file doesn't exist it will be created
local path = system.pathForFile("data.db", system.DocumentsDirectory)
db = sqlite3.open( path )
--Handle the applicationExit event to close the db
local function onSystemEvent( event )
if( event.type == "applicationExit" ) then
db:close()
end
end
--Setup the table if it doesn't exist
local tablesetup = [[CREATE TABLE IF NOT EXISTS test (id INTEGER PRIMARY KEY, content, content2);]]
print(tablesetup)
db:exec( tablesetup )
--Add rows with a auto index in 'id'. You don't need to specify a set of values because we're populating all of them
local testvalue = {}
testvalue[1] = 'Hello'
testvalue[2] = 'World'
testvalue[3] = 'Lua'
local tablefill =[[INSERT INTO test VALUES (NULL, ']]..testvalue[1]..[[',']]..testvalue[2]..[['); ]]
local tablefill2 =[[INSERT INTO test VALUES (NULL, ']]..testvalue[2]..[[',']]..testvalue[1]..[['); ]]
local tablefill3 =[[INSERT INTO test VALUES (NULL, ']]..testvalue[1]..[[',']]..testvalue[3]..[['); ]]
db:exec( tablefill )
db:exec( tablefill2 )
db:exec( tablefill3 )
--print the sqlite version to the terminal
print( "version " .. sqlite3.version() )
--print all the table contents
for row in db:nrows("SELECT * FROM test") do
local text = row.content.." "..row.content2
local t = display.newText(text, 20, 30 * row.id, null, 16)
t:setFillColor( 1, 0, 1 )
end
--setup the system listener to catch applicationExit
Runtime:addEventListener( "system", onSystemEvent )
You can also use third party libraries (or batteries) for creating connections. One such library common in use is LuaSQL.
A thread on the Corona community shows you how to establish connection using LuaSQL for MySQL servers.
local luasql = require "luasql.mysql";
env = assert((luasql.mysql()), "Uh oh, couldn't load driver")
conn = assert(env:connect("database","username","password","localhost"), "Oops!!")
cur = assert (conn:execute ("SELECT * from table_1" ))
row = cur:fetch ({}, "a")
while row do
print ("\n------ new row ---------\n")
table.foreach (row, print)
row = cur:fetch (row, "a")
end
cur:close()
conn:close()
env:close()
Related
I'm creating an external table using a CSV stored in an Azure Data Lake Storage and populating the table using Polybase in SQL Server.
However, I ran into this problem and figured it may be due to the fact that in one particular column there are double quotes present within the string, and the string delimiter has been specified as " in Polybase (STRING_DELIMITER = '"').
HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: HadoopExecutionException: Could not find a delimiter after string delimiter
Example:
I have done quite an extensive research in this and found that this issue has been around for years but yet to see any solutions given.
Any help will be appreciated.
I think the easiest way to fix this up because you are in charge of the .csv creation is to use a delimiter which is not a comma and leave off the string delimiter. Use a separator which you know will not appear in the file. I've used a pipe in my example, and I clean up the string once it is imported in to the database.
A simple example:
IF EXISTS ( SELECT * FROM sys.external_tables WHERE name = 'delimiterWorking' )
DROP EXTERNAL TABLE delimiterWorking
GO
IF EXISTS ( SELECT * FROM sys.tables WHERE name = 'cleanedData' )
DROP TABLE cleanedData
GO
IF EXISTS ( SELECT * FROM sys.external_file_formats WHERE name = 'ff_delimiterWorking' )
DROP EXTERNAL FILE FORMAT ff_delimiterWorking
GO
CREATE EXTERNAL FILE FORMAT ff_delimiterWorking
WITH (
FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS (
FIELD_TERMINATOR = '|',
--STRING_DELIMITER = '"',
FIRST_ROW = 2,
ENCODING = 'UTF8'
)
);
GO
CREATE EXTERNAL TABLE delimiterWorking (
id INT NOT NULL,
body VARCHAR(8000) NULL
)
WITH (
LOCATION = 'yourLake/someFolder/delimiterTest6.txt',
DATA_SOURCE = ds_azureDataLakeStore,
FILE_FORMAT = ff_delimiterWorking,
REJECT_TYPE = VALUE,
REJECT_VALUE = 0
);
GO
SELECT *
FROM delimiterWorking
GO
-- Fix up the data
CREATE TABLE cleanedData
WITH (
CLUSTERED COLUMNSTORE INDEX,
DISTRIBUTION = ROUND_ROBIN
)
AS
SELECT
id,
body AS originalCol,
SUBSTRING ( body, 2, LEN(body) - 2 ) cleanBody
FROM delimiterWorking
GO
SELECT *
FROM cleanedData
My results:
String Delimiter issue can be avoided if you have the Data lake flat file converted to Parquet format.
Input:
"ID"
"NAME"
"COMMENTS"
"1"
"DAVE"
"Hi "I am Dave" from"
"2"
"AARO"
"AARO"
Steps:
1 Convert Flat file to Parquet format [Using Azure Data factory]
2 Create External File format in Data Lake [Assuming Master key, Scope credentials available]
CREATE EXTERNAL FILE FORMAT PARQUET_CONV
WITH (FORMAT_TYPE = PARQUET,
DATA_COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'
);
3 Create External Table with FILE_FORMAT = PARQUET_CONV
Output:
I believe this is the best option as Microsoft don't have an solution currently to handle this string delimiter occurring with in the data for External table
Quite new to coding, I am stuck with this code.It prints out the counts in the command terminal but doesn't create the database file in SQLite. The sqlite version I am using is 3.10.1
import sqlite3
conn = sqlite3.connect('mydb.sqlite')
cur = conn.cursor()
cur.execute('''
DROP TABLE IF EXISTS Counts''')
cur.execute('''CREATE TABLE Counts (org TEXT, count INTEGER)''')
fname = raw_input('Enter file name: ')
if ( len(fname) < 1 ) : fname = 'abc.txt'
fh = open(fname)
for line in fh:
if not (line.startswith('From: ')) : continue
pieces = line.split()
org = pieces[1].split('#')[1]
print org
cur.execute('''SELECT count FROM Counts WHERE org = ? ''', (org, ))
row = cur.fetchone()
if row is None:
cur.execute('''INSERT INTO Counts (org, count) VALUES ( ?, 1 )''', ( org, ) )
else :
cur.execute('''UPDATE Counts SET count=count+1 WHERE org = ?''',
(org, ))
# This statement commits outstanding changes to disk each
# time through the loop - the program can be made faster
# by moving the commit so it runs only after the loop completes
conn.commit()
sqlstr = '''SELECT org, count FROM Counts ORDER BY count DESC LIMIT 10'''
print "Counts:"
for row in cur.execute(sqlstr) :
print str(row[0]), row[1]
cur.close()
I would like to export a data frame to mssql table.I used the code below but I would like to set the destination and not only the server and table name.I have a few DBs inside the server,how can i save the table in one of them?
df<-read.csv(file.choose(),header = T,sep= T)
DB= odbcConnect(dsn ='R_BISRV',uid = 'XXXX', pwd = 'XXX')
sqlSave(DB, df, tablename = 'Tanya', rownames = F,append = T)
close(DB)
I figured it out:
Data base name should be in the odbcDriverConnect() and table name in the sqlSave()
channel <- odbcDriverConnect('driver={SQL Server};server=YYY;database=YY;port=1433;
uid=XX;pwd=XXX')
# Client systems use TCP 1433 to connect to the database engine
sqlSave(channel = channel,dat = df, rownames = TRUE, tablename = "Tanya")
I am using database to keep the highscore record. On storyboard scene change i make file to display highscore. I make database first, then table, then retrieve the record. Check if now made score is more than the existing in database. Update the first existing record.
Its working perfectly fine on the simulator, but on device it stucks on the previous scene and never changes the scene to the highscore file.
It was changing scene before database was implemented.
local myData = require("myData")
-----------------------------------------------------------------------------
-- DATABASE
-- SQLite
-----------------------------------------------------------------------------
--Include sqlite
require "sqlite3"
local path = system.pathForFile("myDataBaseASDF.db", system.DocumentsDirectory)
db = sqlite3.open( path )
--Handle the applicationExit event to close the db
local function onSystemEvent( event )
if( event.type == "applicationExit" ) then
db:close()
end
end
local tablesetup = [[CREATE TABLE IF NOT EXISTS highScoreClassic (id INTEGER PRIMARY KEY, Time, Taps);]]
print("DB Created")
db:exec( tablesetup )
local dbRows = 0
local dbTaps = 0
local dbTime = "00:00:00"
--print all the table contents
for row in db:nrows("SELECT * FROM highScoreClassic WHERE id=1;") do
dbRows = dbRows + 1
dbTaps = row.Taps
dbTime = row.Time
print("Row Taken")
end
function isHighscore()
if myData.currentScore > dbTime then
return true
elseif myData.currentScore == dbTime and myData.taps > dbTaps then
return true
elseif myData.currentScore == dbTime and myData.taps < dbTaps then
return false
else return false end
end
function saveToDataBase()
print("updating")
local q = [[UPDATE highScoreClassic SET Time=']]..myData.currentScore..[[', Taps=]]..myData.taps..[[ WHERE id=1;]]
db:exec( q )
end
function saveToDataBaseFirstTime()
print("inserting")
local tablefill =[[INSERT INTO highScoreClassic VALUES (NULL, ']]..myData.currentScore..[[',']]..myData.taps..[['); ]]
db:exec( tablefill )
end
if ( dbRows > 0 ) then
if ( isHighscore() ) then
print(dbTime, dbTaps)
myData.highScore = true
saveToDataBase()
else
myData.highScore = false
print("No HighScore")
end
else
saveToDataBaseFirstTime()
end
--setup the system listener to catch applicationExit
Runtime:addEventListener( "system", onSystemEvent )
be careful with case sensitive stuff on your device. It's not case sensitive in simulator, but it is on your device.
also make sure the path is correct for your device.
file in simulator DocumentsDirectory wont be included in your device. you have to create the file again.
I am trying to create a Sqlite3 database with luasql. After I require luasql.sqlite3, how do I create the database on a file?
Also, I can't seem to find the manual for luasql. Is it available anywhere?
SQLLite will create the db automatically if it does not exist.
Here is a set of Sample functions for using it in Lua (ignore the fh functions they are just internal to the program I use).
require 'luasql.sqlite3'
function opendb(dbname)
-- Check for Settings Database and create if needed
local db = fhGetPluginDataFileName()
local dbenv = assert (luasql.sqlite3())
-- connect to data source, if the file does not exist it will be created
dbcon = assert (dbenv:connect(db))
-- check table for page list
checkTable(dbcon,'pagelist',
[[CREATE TABLE pagelist(filename varchar(500), md5hash varchar(32),UNIQUE (filename))
]])
-- create table for settings
checkTable(dbcon,'settings',
[[CREATE TABLE settings(key varchar(20), directory varchar(500),
host varchar(500), folder varchar(50), userid varchar(50), password varchar(50), UNIQUE (key))
]])
return dbenv,dbcon
end
function checkTable(dbcon,table,createString)
local sql = string.format([[SELECT count(name) as count FROM sqlite_master WHERE type='table' AND name='%s']],table)
local cur = assert(dbcon:execute(sql))
local rowcount = cur:fetch (row, "a")
cur:close()
if tonumber(rowcount) == 0 then
-- Table not found create it
res,err = assert(dbcon:execute(createString))
end
end
function closedb(dbenv,dbcon)
dbcon:close()
dbenv:close()
end
function loadSettings(dbcon)
local sql = [[SELECT * FROM settings]]
local cur,err = assert(dbcon:execute(sql))
local row = cur:fetch({},'a')
cur:close()
if row then
return row
else
-- return default values
return {
directory = fhGetContextInfo('CI_PROJECT_PUBLIC_FOLDER')..'\\FH Website',
host = 'websitehost',
folder = '/',
userid = 'user',
password = 'password',
new = 'yes'
}
end
end
function saveSettings(dbcon,settings)
-- Check for Settings
if settings.new == 'yes' then
-- Create
sql = string.format([[insert into settings (directory, host, folder, userid, password) Values('%s','%s','%s','%s','%s')]],settings.directory,settings.host,settings.folder,settings.userid,settings.password)
else
-- Update
sql = string.format([[update settings set directory = '%s', host = '%s',folder = '%s',userid = '%s', password = '%s']],settings.directory,settings.host,settings.folder,settings.userid,settings.password)
end
local res = assert(dbcon:execute(sql))
end