How to run ms access database in jar using java 8 ucanaccess - database

I have develop a program in java using eclipse, its a JTable that would connect to my ms access database, all of the data from my ms access table will be shown in my JTable in a click of a button. My program works fine every time i run it in eclipse. My problem is that, when i export it and make a runnable jar file and click that created jar file this message appears:
java.sql.SQLException: No suitable driver found for jdbc:ucanaccess://SUGGESTION_SOURCE_FODEL/suggestionTableFinal.mdb
SUGGESTION_SOURCE_FODEL is the folder where my ms access database table is located and suggestionTableFinal is my ms access datbase table. Any help would be much appreciated! My code are as follow:
public void()
{
try
{
reloadDataSuggestion();
model = new DefaultTableModel(data,columnNames);
mainJTableSuggestion.setModel(model);
Dimension tableSize = mainJTableSuggestion.getPreferredSize();
mainJTableSuggestion.getColumn("SUGGESTIONS").setPreferredWidth(Math.round((tableSize.width - 220)* 1.20f));
DefaultTableCellRenderer centerRenderer = new DefaultTableCellRenderer();
centerRenderer.setHorizontalAlignment( JLabel.CENTER );
mainJTableSuggestion.getColumnModel().getColumn(0).setCellRenderer( centerRenderer );
TableRowSorter<TableModel> sorter = new TableRowSorter<TableModel>(model);
mainJTableSuggestion.setRowSorter(sorter);
st1.close();
rs1.close();
}
catch(Exception e1)
{
JOptionPane.showMessageDialog(null, e1 + "This is my Error");
}
}
public void reloadDataSuggestion() throws ClassNotFoundException, SQLException
{
try
{
columnNames.clear();
data.clear();
String DBPAD = "SUGGESTION_SOURCE_FODEL/suggestionTableFinal.mdb";
String DB = "jdbc:ucanaccess://" + DBPAD;
con1 = DriverManager.getConnection(DB, "", "");
st1 = con1.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE,ResultSet.CONCUR_UPDATABLE);
String sql = ("SELECT * FROM suggestionTableFinal");
rs1 = st1.executeQuery(sql);
ResultSetMetaData rsmd = rs1.getMetaData();
int column = rsmd.getColumnCount();
columnNames.addElement("Submitted");
columnNames.addElement("ClientNames");
columnNames.addElement("SUGGESTIONS");
columnNames.addElement("DateActed");
columnNames.addElement("Number");
columnNames.addElement("SuggestionAction");
while(rs1.next())
{
Vector<Object> row = new Vector<Object>(column);
for(int i=1; i<=column; i++)
{
row.addElement(rs1.getObject(i));
}
data.addElement(row);
}
showDataToTextFieldsFromJTableConnectedToDataBase1();
rs1.close();
st1.close();
totalNumber = 0;
totalNumber1 = 0;
}
catch(Exception e1)
{
final JOptionPane pane = new JOptionPane("<html>" +"<font color=\"#FF0000\">" + "<b><html><span style='font-size:1.2em'>" + e1);
final JDialog d = pane.createDialog((JFrame)null, "TitleFinal1");
d.setLocationRelativeTo(null);
d.setAlwaysOnTop(true);
d.setVisible(true);
}
}

Related

Script task in SSIS package is executing but not performing the action

I have the two SSIS packages which basically has two action like below
First it truncates the contents of the table and then it executes the script task like basically call an API and inserts the response in to the table
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
public async void Main()
{
try
{
var sqlConn = new System.Data.SqlClient.SqlConnection();
ConnectionManager cm = Dts.Connections["SurplusMouse_ADONET"];
string serviceUrl = Dts.Variables["$Project::RM_ServiceUrl"].Value.ToString();
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
HttpClient client = new HttpClient();
client.BaseAddress = new Uri(serviceUrl);
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
string APIUrl = string.Format(serviceUrl + "/gonogo");
var response = await client.GetAsync(APIUrl);
if (response.IsSuccessStatusCode)
{
var result = await response.Content.ReadAsStringAsync();
try
{
sqlConn = (System.Data.SqlClient.SqlConnection)cm.AcquireConnection(Dts.Transaction);
const string query = #"INSERT INTO [dbo].[RM_Approved_Room_State]
(APPROVED_ROOM_STATEID,SOURCE_ROOMID,DEST_ROOMID,ENTITY_TYPEID)
SELECT id, sourceRoomRefId, destinationRoomRefId,entityRefId
FROM OPENJSON(#json)
WITH (
id int,
sourceRoomRefId int,
destinationRoomRefId int,
entityRefId int
) j;";
using (var sqlCmd = new System.Data.SqlClient.SqlCommand(query, sqlConn))
{
sqlCmd.Parameters.Add("#json", SqlDbType.NVarChar, -1).Value = result;
await sqlCmd.ExecuteNonQueryAsync();
}
}
catch (Exception ex)
{
Dts.TaskResult = (int)ScriptResults.Failure;
}
finally
{
if (sqlConn != null)
cm.ReleaseConnection(sqlConn);
}
}
}
catch (Exception ex)
{
Dts.TaskResult = (int)ScriptResults.Failure;
}
}
#region ScriptResults declaration
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
} }
Similar to the above package I have another one which does more likely does insert records into different table the response from a different endpoint. When I execute the packages locally/ execute them separately after deploying it in to the server it works fine. But when I add them in to the SQL Server Agent Job like below and run them on a schedule
The Jobs run successfully and dont show any errors but I can see only one table with data from one package but the other one truncates the records but I dont think the script task is getting executed / I dont see any records inserted. I dont think there are any issues with access because when I run them seperate manually the data are getting inserted, Just when it is running on a schedule it is not working as expected. Any idea what could be happening here.. Any help is greatly appreciated

How to use scope_identity in jdbc [duplicate]

I want to INSERT a record in a database (which is Microsoft SQL Server in my case) using JDBC in Java. At the same time, I want to obtain the insert ID. How can I achieve this using JDBC API?
If it is an auto generated key, then you can use Statement#getGeneratedKeys() for this. You need to call it on the same Statement as the one being used for the INSERT. You first need to create the statement using Statement.RETURN_GENERATED_KEYS to notify the JDBC driver to return the keys.
Here's a basic example:
public void create(User user) throws SQLException {
try (
Connection connection = dataSource.getConnection();
PreparedStatement statement = connection.prepareStatement(SQL_INSERT,
Statement.RETURN_GENERATED_KEYS);
) {
statement.setString(1, user.getName());
statement.setString(2, user.getPassword());
statement.setString(3, user.getEmail());
// ...
int affectedRows = statement.executeUpdate();
if (affectedRows == 0) {
throw new SQLException("Creating user failed, no rows affected.");
}
try (ResultSet generatedKeys = statement.getGeneratedKeys()) {
if (generatedKeys.next()) {
user.setId(generatedKeys.getLong(1));
}
else {
throw new SQLException("Creating user failed, no ID obtained.");
}
}
}
}
Note that you're dependent on the JDBC driver as to whether it works. Currently, most of the last versions will work, but if I am correct, Oracle JDBC driver is still somewhat troublesome with this. MySQL and DB2 already supported it for ages. PostgreSQL started to support it not long ago. I can't comment about MSSQL as I've never used it.
For Oracle, you can invoke a CallableStatement with a RETURNING clause or a SELECT CURRVAL(sequencename) (or whatever DB-specific syntax to do so) directly after the INSERT in the same transaction to obtain the last generated key. See also this answer.
Create Generated Column
String generatedColumns[] = { "ID" };
Pass this geneated Column to your statement
PreparedStatement stmtInsert = conn.prepareStatement(insertSQL, generatedColumns);
Use ResultSet object to fetch the GeneratedKeys on Statement
ResultSet rs = stmtInsert.getGeneratedKeys();
if (rs.next()) {
long id = rs.getLong(1);
System.out.println("Inserted ID -" + id); // display inserted record
}
When encountering an 'Unsupported feature' error while using Statement.RETURN_GENERATED_KEYS, try this:
String[] returnId = { "BATCHID" };
String sql = "INSERT INTO BATCH (BATCHNAME) VALUES ('aaaaaaa')";
PreparedStatement statement = connection.prepareStatement(sql, returnId);
int affectedRows = statement.executeUpdate();
if (affectedRows == 0) {
throw new SQLException("Creating user failed, no rows affected.");
}
try (ResultSet rs = statement.getGeneratedKeys()) {
if (rs.next()) {
System.out.println(rs.getInt(1));
}
rs.close();
}
Where BATCHID is the auto generated id.
I'm hitting Microsoft SQL Server 2008 R2 from a single-threaded JDBC-based application and pulling back the last ID without using the RETURN_GENERATED_KEYS property or any PreparedStatement. Looks something like this:
private int insertQueryReturnInt(String SQLQy) {
ResultSet generatedKeys = null;
int generatedKey = -1;
try {
Statement statement = conn.createStatement();
statement.execute(SQLQy);
} catch (Exception e) {
errorDescription = "Failed to insert SQL query: " + SQLQy + "( " + e.toString() + ")";
return -1;
}
try {
generatedKey = Integer.parseInt(readOneValue("SELECT ##IDENTITY"));
} catch (Exception e) {
errorDescription = "Failed to get ID of just-inserted SQL query: " + SQLQy + "( " + e.toString() + ")";
return -1;
}
return generatedKey;
}
This blog post nicely isolates three main SQL Server "last ID" options:
http://msjawahar.wordpress.com/2008/01/25/how-to-find-the-last-identity-value-inserted-in-the-sql-server/ - haven't needed the other two yet.
Instead of a comment, I just want to answer post.
Interface java.sql.PreparedStatement
columnIndexes « You can use prepareStatement function that accepts columnIndexes and SQL statement.
Where columnIndexes allowed constant flags are Statement.RETURN_GENERATED_KEYS1 or Statement.NO_GENERATED_KEYS[2], SQL statement that may contain one or more '?' IN parameter placeholders.
SYNTAX «
Connection.prepareStatement(String sql, int autoGeneratedKeys)
Connection.prepareStatement(String sql, int[] columnIndexes)
Example:
PreparedStatement pstmt =
conn.prepareStatement( insertSQL, Statement.RETURN_GENERATED_KEYS );
columnNames « List out the columnNames like 'id', 'uniqueID', .... in the target table that contain the auto-generated keys that should be returned. The driver will ignore them if the SQL statement is not an INSERT statement.
SYNTAX «
Connection.prepareStatement(String sql, String[] columnNames)
Example:
String columnNames[] = new String[] { "id" };
PreparedStatement pstmt = conn.prepareStatement( insertSQL, columnNames );
Full Example:
public static void insertAutoIncrement_SQL(String UserName, String Language, String Message) {
String DB_URL = "jdbc:mysql://localhost:3306/test", DB_User = "root", DB_Password = "";
String insertSQL = "INSERT INTO `unicodeinfo`( `UserName`, `Language`, `Message`) VALUES (?,?,?)";
//"INSERT INTO `unicodeinfo`(`id`, `UserName`, `Language`, `Message`) VALUES (?,?,?,?)";
int primkey = 0 ;
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
Connection conn = DriverManager.getConnection(DB_URL, DB_User, DB_Password);
String columnNames[] = new String[] { "id" };
PreparedStatement pstmt = conn.prepareStatement( insertSQL, columnNames );
pstmt.setString(1, UserName );
pstmt.setString(2, Language );
pstmt.setString(3, Message );
if (pstmt.executeUpdate() > 0) {
// Retrieves any auto-generated keys created as a result of executing this Statement object
java.sql.ResultSet generatedKeys = pstmt.getGeneratedKeys();
if ( generatedKeys.next() ) {
primkey = generatedKeys.getInt(1);
}
}
System.out.println("Record updated with id = "+primkey);
} catch (InstantiationException | IllegalAccessException | ClassNotFoundException | SQLException e) {
e.printStackTrace();
}
}
I'm using SQLServer 2008, but I have a development limitation: I cannot use a new driver for it, I have to use "com.microsoft.jdbc.sqlserver.SQLServerDriver" (I cannot use "com.microsoft.sqlserver.jdbc.SQLServerDriver").
That's why the solution conn.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS) threw a java.lang.AbstractMethodError for me.
In this situation, a possible solution I found is the old one suggested by Microsoft:
How To Retrieve ##IDENTITY Value Using JDBC
import java.sql.*;
import java.io.*;
public class IdentitySample
{
public static void main(String args[])
{
try
{
String URL = "jdbc:microsoft:sqlserver://yourServer:1433;databasename=pubs";
String userName = "yourUser";
String password = "yourPassword";
System.out.println( "Trying to connect to: " + URL);
//Register JDBC Driver
Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver").newInstance();
//Connect to SQL Server
Connection con = null;
con = DriverManager.getConnection(URL,userName,password);
System.out.println("Successfully connected to server");
//Create statement and Execute using either a stored procecure or batch statement
CallableStatement callstmt = null;
callstmt = con.prepareCall("INSERT INTO myIdentTable (col2) VALUES (?);SELECT ##IDENTITY");
callstmt.setString(1, "testInputBatch");
System.out.println("Batch statement successfully executed");
callstmt.execute();
int iUpdCount = callstmt.getUpdateCount();
boolean bMoreResults = true;
ResultSet rs = null;
int myIdentVal = -1; //to store the ##IDENTITY
//While there are still more results or update counts
//available, continue processing resultsets
while (bMoreResults || iUpdCount!=-1)
{
//NOTE: in order for output parameters to be available,
//all resultsets must be processed
rs = callstmt.getResultSet();
//if rs is not null, we know we can get the results from the SELECT ##IDENTITY
if (rs != null)
{
rs.next();
myIdentVal = rs.getInt(1);
}
//Do something with the results here (not shown)
//get the next resultset, if there is one
//this call also implicitly closes the previously obtained ResultSet
bMoreResults = callstmt.getMoreResults();
iUpdCount = callstmt.getUpdateCount();
}
System.out.println( "##IDENTITY is: " + myIdentVal);
//Close statement and connection
callstmt.close();
con.close();
}
catch (Exception ex)
{
ex.printStackTrace();
}
try
{
System.out.println("Press any key to quit...");
System.in.read();
}
catch (Exception e)
{
}
}
}
This solution worked for me!
I hope this helps!
You can use following java code to get new inserted id.
ps = con.prepareStatement(query, Statement.RETURN_GENERATED_KEYS);
ps.setInt(1, quizid);
ps.setInt(2, userid);
ps.executeUpdate();
ResultSet rs = ps.getGeneratedKeys();
if (rs.next()) {
lastInsertId = rs.getInt(1);
}
It is possible to use it with normal Statement's as well (not just PreparedStatement)
Statement statement = conn.createStatement();
int updateCount = statement.executeUpdate("insert into x...)", Statement.RETURN_GENERATED_KEYS);
try (ResultSet generatedKeys = statement.getGeneratedKeys()) {
if (generatedKeys.next()) {
return generatedKeys.getLong(1);
}
else {
throw new SQLException("Creating failed, no ID obtained.");
}
}
Most others have suggested to use JDBC API for this, but personally, I find it quite painful to do with most drivers. When in fact, you can just use a native T-SQL feature, the OUTPUT clause:
try (
Statement s = c.createStatement();
ResultSet rs = s.executeQuery(
"""
INSERT INTO t (a, b)
OUTPUT id
VALUES (1, 2)
"""
);
) {
while (rs.next())
System.out.println("ID = " + rs.getLong(1));
}
This is the simplest solution for SQL Server as well as a few other SQL dialects (e.g. Firebird, MariaDB, PostgreSQL, where you'd use RETURNING instead of OUTPUT).
I've blogged about this topic more in detail here.
With Hibernate's NativeQuery, you need to return a ResultList instead of a SingleResult, because Hibernate modifies a native query
INSERT INTO bla (a,b) VALUES (2,3) RETURNING id
like
INSERT INTO bla (a,b) VALUES (2,3) RETURNING id LIMIT 1
if you try to get a single result, which causes most databases (at least PostgreSQL) to throw a syntax error. Afterwards, you may fetch the resulting id from the list (which usually contains exactly one item).
In my case ->
ConnectionClass objConnectionClass=new ConnectionClass();
con=objConnectionClass.getDataBaseConnection();
pstmtGetAdd=con.prepareStatement(SQL_INSERT_ADDRESS_QUERY,Statement.RETURN_GENERATED_KEYS);
pstmtGetAdd.setString(1, objRegisterVO.getAddress());
pstmtGetAdd.setInt(2, Integer.parseInt(objRegisterVO.getCityId()));
int addId=pstmtGetAdd.executeUpdate();
if(addId>0)
{
ResultSet rsVal=pstmtGetAdd.getGeneratedKeys();
rsVal.next();
addId=rsVal.getInt(1);
}
If you are using Spring JDBC, you can use Spring's GeneratedKeyHolder class to get the inserted ID.
See this answer...
How to get inserted id using Spring Jdbctemplate.update(String sql, obj...args)
If you are using JDBC (tested with MySQL) and you just want the last inserted ID, there is an easy way to get it. The method I'm using is the following:
public static Integer insert(ConnectionImpl connection, String insertQuery){
Integer lastInsertId = -1;
try{
final PreparedStatement ps = connection.prepareStatement(insertQuery);
ps.executeUpdate(insertQuery);
final com.mysql.jdbc.PreparedStatement psFinal = (com.mysql.jdbc.PreparedStatement) ps;
lastInsertId = (int) psFinal.getLastInsertID();
connection.close();
} catch(SQLException ex){
System.err.println("Error: "+ex);
}
return lastInsertId;
}
Also, (and just in case) the method to get the ConnectionImpl is the following:
public static ConnectionImpl getConnectionImpl(){
ConnectionImpl conexion = null;
final String dbName = "database_name";
final String dbPort = "3306";
final String dbIPAddress = "127.0.0.1";
final String connectionPath = "jdbc:mysql://"+dbIPAddress+":"+dbPort+"/"+dbName+"?autoReconnect=true&useSSL=false";
final String dbUser = "database_user";
final String dbPassword = "database_password";
try{
conexion = (ConnectionImpl) DriverManager.getConnection(connectionPath, dbUser, dbPassword);
}catch(SQLException e){
System.err.println(e);
}
return conexion;
}
Remember to add the connector/J to the project referenced libraries.
In my case, the connector/J version is the 5.1.42. Maybe you will have to apply some changes to the connectionPath if you want to use a more modern version of the connector/J such as with the version 8.0.28.
In the file, remember to import the following resources:
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import com.mysql.jdbc.ConnectionImpl;
Hope this will be helpful.
Connection cn = DriverManager.getConnection("Host","user","pass");
Statement st = cn.createStatement("Ur Requet Sql");
int ret = st.execute();

Create Database in PervasiveSQL from Command Line

How do I create a database in PervasiveSQL using the command line.
I know how to do it via Control Center, but I would rather create it via the command line. I am working to automate the standup of a PervasiveSQL box for a project I am working on. I have the server install happening silently and I am adjusting the Server Configuration using a RegKey import.
Now i just need to script the creation of the database. The new database will use existing database files which are already copied to the server.
In the documentation I am using found here: there is a utility called dbMaint (page 264) which seems like it would do the job, but I do not seem to have that tool on my server.
Thank you in advance for your help.
dbMaint is only provided for PSQL on Linux. There is a way to write a utility using the Distributed Tuning Interface (DTI) or Distributed Tuning Object (DTO) to create the database. I can't link to the PSQL documentation but there is a PSQL_DTI_GUIDE.pdf and PSQL_DTO_Guide.pdf in the PSQL v11 documentation download that describes how to use those APIs.
Found a C# sample I put together a while back. The Pervasive DTO library will need to be added as a reference. It's a COM object. The simple sample is:
using System;
using DTOLib;
namespace dtoTest
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string compName = null;
string userName = null;
string password = null;
string dbname = null;
string ddflocation = null;
string datalocation = null;
dtoDbFlags dbflags = DTOLib.dtoDbFlags.dtoDbFlagDefault;
DTOLib.dtoResult result;
if (args.LongLength < 1)
{
Console.WriteLine("Invalid options.\n");
Console.WriteLine("Usage: dtoDBN.EXE <computername> <username> <password> <dbname> <ddf location> <data location> <DBFlage>");
Console.WriteLine("NOTE: locations must be relative to the computer where the PSQL engine is running.");
Console.WriteLine("DB Flags must be passed as integer in this example with these values: ");
Console.WriteLine(" P_DBFLAG_BOUND = 1; (* bound database - must have P_DBFLAG_CREATE_DDF too *)");
Console.WriteLine(" P_DBFLAG_RI = 2; (*relational integrity *)");
Console.WriteLine(" P_DBFLAG_CREATE_DDF = 4; (*create ddf flag *)");
Console.WriteLine(" P_DBFLAG_NA = 2147483648; (*not applicable *)");
Console.WriteLine(" P_DBFLAG_DEFAULT = (P_DBFLAG_BOUND or P_DBFLAG_RI); ");
return;
}
if (args.LongLength == 7)
{
compName = args[0].ToString();
userName = args[1].ToString();
password = args[2].ToString();
dbname = args[3].ToString();
ddflocation = args[4].ToString();
datalocation = args[5].ToString();
dbflags = (dtoDbFlags)Convert.ToInt32(args[6]);
}
Console.WriteLine("Create Pervasive Database using DTO and C#");
DtoSession mDtoSession = new DTOLib.DtoSession();
try
{
result = mDtoSession.Connect(compName, userName, password);
if (result != 0)
{
Console.WriteLine("Error connecting to server. Error code:");
}
else
{
//Create a Database name here.
DtoDatabase db = new DtoDatabase();
db.Name = dbname;
db.DdfPath = ddflocation;
db.DataPath = datalocation;
db.Flags = dbflags;
result = mDtoSession.Databases.Add(db);
if (result !=0)
{
Console.WriteLine(string.Format("Error creating the datbase ({0}). Error code: {1}", dbname, result.ToString()));
}
else
{
Console.WriteLine(string.Format("Database ({0}) created. ", dbname));
}
result = mDtoSession.Disconnect();
Console.ReadLine();
}
}
catch (Exception e1)
{
Console.WriteLine(e1.Message.ToString());
}
}
}
}

cannot bind on a new display member combobox c#

When I run my program on my machine it works totally fine but when I run the code from other machine, I get an Error that says "Cannot bind on a new display member"? I just don't understand why? Here is my code.
private DataTable effectivity, filesTable, cgmFiles;
private void FillEffectivity()
{
cbEffectivity.ComboBox.DataSource = null;
effectivity = FillDataTable("SELECT * FROM Projects ORDER BY Project ASC");
cbEffectivity.ComboBox.DataSource = effectivity;
cbEffectivity.ComboBox.DisplayMember = "project";
cbEffectivity.ComboBox.ValueMember = "RecNo";
}
private DataTable FillTable(string sql)
{
DataTable table = new DataTable();
try
{
using (OleDbDataAdapter da = new OleDbDataAdapter(sql, cn))
{
da.Fill(table);
}
}
catch (Exception ex)
{
if (ex.Message == "Not a valid password.")
{
SetPassword();
return FillTable(sql);
}
return new DataTable();
}
finally
{
cn.Close();
}
return table;
}
It seems like problem in your query instead of effectivity = FillDataTable("SELECT * FROM Projects ORDER BY Project ASC"); try FillDataTable("SELECT * FROM Projects ORDER BY project ASC");
also it is recommended that dont use * while selecting it will affect on performance.
try this
FillDataTable("SELECT project FROM Projects ORDER BY project ASC");

Different ways of performing bulk insert into database from a java application

I am looking for different ways of performing bulk insert into database (e.g. SQL Server 2012) from a Java application. I need to insert lot of entities into database very efficiently without making as many calls to database as there are entities.
My requirement is to perform a bulk insert of entities, where an insert of entity in database could involve inserting data into one or more tables. The following are the two ways which I can think of:
Dynamically generate a batch of SQL statements and execute it against the database by making use of native JDBC support.
Construct XML representation of all the entities and then invoke a stored procedure by passing the generated XML. The stored procedure takes care of parsing the XML and inserting the entities to database.
I am new to Java and not having enough knowledge of available frameworks. IMO, the above two approaches seems to be very naive and not leveraging the available frameworks. I am requesting experts to share different ways of achieving bulk insert along with its pros and cons. I am open to MyBatis, Spring-MyBatis, Spring-JDBC, JDBC, etc which solves the problem in an efficient manner.
Thanks.
I have a demo ,JDBC batch processing
file:demo.txt
The content
1899942 ,demo1
1899944 ,demo2
1899946 ,demo3
1899948 ,demo4
Insert the data reads the file content
my code:
public class Test2 {
public static void main(String[] args) {
long start = System.currentTimeMillis();
String sql = "insert into mobile_place(number,place) values(?,?)";
int count=0;
PreparedStatement pstmt = null;
Connection conn = JDBCUtil.getConnection();
try {
pstmt = conn.prepareStatement(sql);
InputStreamReader is = new InputStreamReader(new FileInputStream(new File("D:/CC.txt")),"utf-8");
BufferedReader br = new BufferedReader(is);
conn.setAutoCommit(false);
String s1 = null;
String s2 = null;
while(br.readLine() != null){
count++;
String str = br.readLine().toString().trim();
s1 = str.substring(0, str.indexOf(","));
s2 = str.substring(str.indexOf(",")+1,str.length());
pstmt.setString(1, s1);
pstmt.setString(2, s2);
pstmt.addBatch();
if(count%1000==0){
pstmt.executeBatch();
conn.commit();
conn.close();
conn = JDBCUtil.getConnection();
conn.setAutoCommit(false);
pstmt = conn.prepareStatement(sql);
}
System.out.println("insert "+count+"line");
}
if(count%1000!=0){
pstmt.executeBatch();
conn.commit();
}
long end = System.currentTimeMillis();
System.out.println("Total time spent:"+(end-start));
} catch (Exception e) {
e.printStackTrace();
}finally{
try {
pstmt.close();
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
//getConnection()//get jdbc Connection
public static Connection getConnection(){
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
try {
conn = DriverManager.getConnection(url, userName, password);
} catch (SQLException e) {
e.printStackTrace();
}
return conn;
}
Speak for the first time, I hope I can help
I am the demo above use PreparedStatement [Read data calls a PreparedStatement one-off inserted]
JDBC batch There are 3 ways
1.use PreparedStatement
demo:
try {
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection(o_url, userName, password);
conn.setAutoCommit(false);
String sql = "INSERT adlogs(ip,website,yyyymmdd,hour,object_id) VALUES(?,?,?,?,?)";
PreparedStatement prest = conn.prepareStatement(sql,ResultSet.TYPE_SCROLL_SENSITIVE,ResultSet.CONCUR_READ_ONLY);
for(int x = 0; x < size; x++){
prest.setString(1, "192.168.1.1");
prest.setString(2, "localhost");
prest.setString(3, "20081009");
prest.setInt(4, 8);
prest.setString(5, "11111111");
prest.addBatch();
}
prest.executeBatch();
conn.commit();
conn.close();
} catch (SQLException ex) {
Logger.getLogger(MyLogger.class.getName()).log(Level.SEVERE, null, ex);
}
2.use Statement.addBatch methods
demo:
conn.setAutoCommit(false);
Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_READ_ONLY);
for(int x = 0; x < size; x++){
stmt.addBatch("INSERT INTO adlogs(ip,website,yyyymmdd,hour,object_id) VALUES('192.168.1.3', 'localhost','20081009',8,'23123')");
}
stmt.executeBatch();
conn.commit();
3.Direct use of the Statement
demo:
conn.setAutoCommit(false);
Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE,
ResultSet.CONCUR_READ_ONLY);
for(int x = 0; x < size; x++){
stmt.execute("INSERT INTO adlogs(ip,website,yyyymmdd,hour,object_id) VALUES('192.168.1.3', 'localhost','20081009',8,'23123')");
}
conn.commit();
Using the above method Insert the 100000 pieces of data Time consuming:
method 1:17.844s
method 2:18.421s
method 3:16.359s
MS JDBC versions later than 4.1 have SQLServerBulkCopy class that I assume is equivalent to one available in .Net and theoretically it should work as fast as bcp command line utility.
https://msdn.microsoft.com/en-us/library/mt221490%28v=sql.110%29.aspx
you can custom your code with JDBC,there is no framework support your requirement

Resources