on JpaRepository.save(Entity e) e have negative value as primary key in ms sql server database - sql-server

When i do JpaRepository.save(Entity e) the primary key generated with the help of hibernate sequence is saved as any random value generally starting from -43 or -42.
I am spring a spring boot project with JPA.
Below is my property file:
hibernate.dialect=org.hibernate.dialect.SQLServer2012Dialect
hibernate.hbm2ddl.auto=validate
hibernate.ejb.naming_strategy=org.hibernate.cfg.ImprovedNamingStrategy
hibernate.show_sql=false
hibernate.format_sql=true
This is my entity on which i am calling save. Sequence name - CPU_Responses_Seq is already present in DB
#Entity
#Table(name="CPU_Responses")
public class CPUResponses extends BaseEnity{
/**
*
*/
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(generator="CPUResponseSeq",strategy=GenerationType.SEQUENCE)
#SequenceGenerator(name="CPUResponseSeq",sequenceName="CPU_Responses_Seq")
#Column(name = "Response_ID", nullable=false,updatable=false)
private long responseId;
This is my persistance config class
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages= {"package path"})
#PropertySource("classpath:application.properties")
public class PersistanceConfiguration {
#Autowired
private Environment env;
public Environment getEnv() {
return env;
}
public void setEnv(Environment env) {
this.env = env;
}
#Bean
LocalContainerEntityManagerFactoryBean entityManagerFactory() throws NamingException {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource());
entityManagerFactoryBean.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
entityManagerFactoryBean.setPackagesToScan("entity path");
Properties jpaProperties = new Properties();
//Configures the used database dialect. This allows Hibernate to create SQL
//that is optimized for the used database.
jpaProperties.put("hibernate.dialect", env.getRequiredProperty("hibernate.dialect"));
entityManagerFactoryBean.setJpaProperties(jpaProperties);
return entityManagerFactoryBean;
}
#Bean
public DataSource dataSource() throws NamingException {
JndiObjectFactoryBean bean = new JndiObjectFactoryBean();
bean.setJndiName("java:comp/env/jdbc/CPUDB");
bean.setProxyInterface(DataSource.class);
bean.setLookupOnStartup(false);
bean.afterPropertiesSet();
return (DataSource) bean.getObject();
}
#Bean
JpaTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(entityManagerFactory);
return transactionManager;
}
}
I don't know what is getting wrong. Data is getting saved in DB but with negative primary key. My sequencer in DB have min value as zero so sequencer is correct.
Kindly help

I think this is related to the changes introduced in Hibernate in their sequence generator, try to add
hibernate.id.new_generator_mappings=false
or
spring.jpa.properties.hibernate.id.new_generator_mappings=false
note that the "new generator" is not compatible with previous version so start from a clean database to avoid issues.

Related

How to test fluent migrations with an in-process migration runner and a in memory SQLite database

I have just started to use FluentMigration for my current project. I wrote my first migration but I have some trouble writing a unit test for it.
Here is some sample code:
private ServiceProvider CreateServiceProvider()
{
return new ServiceCollection()
.AddLogging(lb => lb.AddFluentMigratorConsole())
.AddFluentMigratorCore()
.ConfigureRunner(
builder => builder
.AddSQLite()
.WithGlobalConnectionString("Data Source=:memory:;Version=3;New=True;")
.WithMigrationsIn(typeof(MigrationOne).Assembly))
.BuildServiceProvider();
}
private void PerformMigrateUp(IServiceScope scope)
{
var runner = scope.ServiceProvider.GetRequiredService<IMigrationRunner>();
runner.MigrateUp(1);
}
[Test]
public void ShouldHaveTablesAfterMigrateUp()
{
var provider = this.CreateServiceProvider();
using (var scope = provider.CreateScope())
{
this.PerformMigrateUp(scope);
// here I'd like to test if tables have been created in the database by the migration
}
}
I don't know how (or if it is possible) to access the current database connection so I can perform a query. Any suggestions would be helpful. Thanks.
Ok, I found a solution. I have to use the Process method of the runner's processor to perform my own sql query.
It looks like this:
private ServiceProvider CreateServiceProvider()
{
return new ServiceCollection()
.AddLogging(lb => lb.AddFluentMigratorConsole())
.AddFluentMigratorCore()
.ConfigureRunner(
builder => builder
.AddSQLite()
.WithGlobalConnectionString(#"Data Source=:memory:;Version=3;New=True;")
.WithMigrationsIn(typeof(MigrationDate20181026113000Zero).Assembly))
.BuildServiceProvider();
}
[Test]
public void ShouldHaveNewVersionAfterMigrateUp()
{
var serviceProvider = this.CreateServiceProvider();
var scope = serviceProvider.CreateScope();
var runner = scope.ServiceProvider.GetRequiredService<IMigrationRunner>();
runner.MigrateUp(1);
string sqlStatement = "SELECT Description FROM VersionInfo";
DataSet dataSet = runner.Processor.Read(sqlStatement, string.Empty);
Assert.That(dataSet, Is.Not.Null);
Assert.That(dataSet.Tables[0].Rows[0].ItemArray[0], Is.EqualTo("Migration1"));
}
This is an old question but an important one. I find it strange that I couldnt find any documentation on this.
In any case here is my solution which I find to be a bit better as you dont need to rely on the runner. Since you dont need that the options open up hugely for constructor arguments.
Firstly make sure you install Microsoft.Data.Sqlite or you will get a strange error.
SQLite in memory databases exist for as long as the connection does - and 1 database per connection on first glance. Actually though there is a way to share the database between connections as long as at least 1 connection is open at all times according to my experiments. You just need to name it.
https://learn.microsoft.com/en-us/dotnet/standard/data/sqlite/connection-strings#sharable-in-memory
So to begin with I created a connection that will stay open until the test finishes. It will be named using Guid.NewGuid() so that subsequent connections will work as expected.
var dbName = Guid.NewGuid().ToString();
var connectionString = $"Data Source={dbName};Mode=Memory;Cache=Shared";
var connection = new SqliteConnection(connectionString);
connection.Open();
After that the crux of running the migrations is the same as previously answered but the connection string uses the named database:
var sp = services.AddFluentMigratorCore()
.ConfigureRunner(fluentMigratorBuilder => fluentMigratorBuilder
.AddSQLite()
.WithGlobalConnectionString(connectionString)
.ScanIn(AssemblyWithMigrations).For.Migrations()
)
.BuildServiceProvider();
var runner = sp.GetRequiredService<IMigrationRunner>();
runner.MigrateUp();
Here is a class I use to inject a connection factory everywhere that needs to connect to the database for normal execution:
internal class PostgresConnectionFactory : IConnectionFactory
{
private readonly string connectionString;
public PostgresConnectionFactory(string connectionString)
{
this.connectionString = connectionString;
}
public DbConnection Create()
{
return new NpgsqlConnection(connectionString);
}
}
I just replaced this (all hail dependency inversion) with:
internal class InMemoryConnectionFactory : IConnectionFactory
{
private readonly string connectionstring;
public InMemoryConnectionFactory(string connectionstring)
{
this.connectionstring = connectionstring;
}
public DbConnection Create()
{
return new SqliteConnection(connectionstring);
}
}
where the connection string is the same named one I defined above.
Now you can simply use that connection factory anywhere that needs to connect to the same in memory database, and since we can now connect multiple times possibilities for integration testing open up.
Here is the majority of my implementation:
public static IDisposable CreateInMemoryDatabase(Assembly AssemblyWithMigrations, IServiceCollection services = null)
{
if (services == null)
services = new ServiceCollection();
var connectionString = GetSharedConnectionString();
var connection = GetPersistantConnection(connectionString);
MigrateDb(services, connectionString, AssemblyWithMigrations);
services.AddSingleton<IConnectionFactory>(new InMemoryConnectionFactory(connectionString));
return services.BuildServiceProvider()
.GetRequiredService<IDisposableUnderlyingQueryingTool>();
}
private static string GetSharedConnectionString()
{
var dbName = Guid.NewGuid().ToString();
return $"Data Source={dbName};Mode=Memory;Cache=Shared";
}
private static void MigrateDb(IServiceCollection services, string connectionString, Assembly assemblyWithMigrations)
{
var sp = services.AddFluentMigratorCore()
.ConfigureRunner(fluentMigratorBuilder => fluentMigratorBuilder
.AddSQLite()
.WithGlobalConnectionString(connectionString)
.ScanIn(assemblyWithMigrations).For.Migrations()
)
.BuildServiceProvider();
var runner = sp.GetRequiredService<IMigrationRunner>();
runner.MigrateUp();
}
private static IDbConnection GetPersistantConnection(string connectionString)
{
var connection = new SqliteConnection(connectionString);
connection.Open();
return connection;
}
Then here is a sample test:
public Test : IDisposable {
private readonly IDisposable _holdingConnection;
public Test() {
_holdingConnection = CreateInMemoryDatabase(typeof(MyFirstMigration).Assembly);
}
public void Dispose() {
_holdingConnection.Dispose();
}
}
You may notice that the static factory returns a custom interface. Its just an interface that extends the normal tooling I inject to repositories, but also implements IDisposable.
Untested bonus for integration testing where you will have a service collection created via WebApplicationFactory or TestServer etc:
public void AddInMemoryPostgres(Assembly AssemblyWithMigrations)
{
var lifetime = services.BuildServiceProvider().GetService<IHostApplicationLifetime>();
var holdingConnection= InMemoryDatabaseFactory.CreateInMemoryDapperTools(AssemblyWithMigrations, services);
lifetime.ApplicationStopping.Register(() => {
holdingConnection.Dispose();
});
}

Spring boot: Database connection not closing properly

I'm executing queries periodically (by a scheduler) using my Spring Boot application
application.properties
src_mssqlserver_url=jdbc:sqlserver://192.168.0.1;databaseName=Test;
src_mssqlserver_username=tester
src_mssqlserver_password=tester1
src_mssqlserver_driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
Datasource and JdbcTemplate Bean
#Primary
#Bean(name = "src_mssqlserver")
#ConfigurationProperties(prefix = "spring.ds_mssqlserver")
public DataSource srcDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(env.getProperty("src_mssqlserver_driverClassName"));
dataSource.setUrl(env.getProperty("src_mssqlserver_url"));
dataSource.setUsername(env.getProperty("src_mssqlserver_username"));
dataSource.setPassword(env.getProperty("src_mssqlserver_password"));
return dataSource;
}
#Bean(name = "srcJdbcTemplate")
public JdbcTemplate srcJdbcTemplate(#Qualifier("src_mssqlserver") DataSource dsSrcSqlServer) {
return new JdbcTemplate(dsSrcSqlServer);
}
Usage: This method is called from a scheduler with list of items to process (normally 1000 records), this process runs in an hour once.
#Autowired
#Qualifier("srcJdbcTemplate")
private JdbcTemplate srcJdbcTemplate;
public void batchInsertUsers(final List<User> users) {
String queryInsert = "INSERT INTO [User] ([Name]"
+ " , [Created_Date]"
+ " , [Notes])"
+ " VALUES (?, SYSDATETIMEOFFSET(), ?)";
srcJdbcTemplate.batchUpdate(queryInsert, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
User user = users.get(i);
ps.setString(1, user.getName());
ps.setString(2, user.getNotes());
}
#Override
public int getBatchSize() {
return sites.size();
}
});
I'm getting warnings from database administrator that my code keeping too much connections open. Please share some standard and workable way to handle such situation.
Thanks.
DriverManagerDataSource is NOT meant for production, it opens and closes a connection each time it needs one.
Use a connection pool like c3p0DataSource.

Switching from local database to SQL server

So our current code loaded a csv file into a local jdbcTemplate in which then I do some querying. The issue was always performance, and we finally got access to a SQL server that could load the data. Naturally the company gets the guy with basically no database skills to set this up :P
#Autowired
DataSource dataSource;
#RequestMapping("/queryService")
public void queryService(#RequestParam("id")String id)
{
log.info("Creating tables");
jdbcTemplate.execute("DROP TABLE accounts IF EXISTS");
jdbcTemplate.execute("CREATE TABLE accounts(id VARCHAR(255), name VARCHAR(255), Organization__c VARCHAR(255)";
insertBatch(accounts,dataSource);
ArrayList<Account2> filteredaccs = filterAccount(jdbcTemplate);
.
public void insertBatch(ArrayList<Account2> accs, DataSource dataSource) {
List<Map<String, Object>> batchValues = new ArrayList<>(accs.size());
for (Account2 a : accs) {
Map<String, Object> map = new HashMap<>();
map.put("id", a.getId());
map.put("name", a.getName());
map.put("Organization__c", a.getOrganization__c());
batchValues.add(map);
}
SimpleJdbcInsert simpleJdbcInsert = new SimpleJdbcInsert(dataSource).withTableName("accounts");
int[] ints = simpleJdbcInsert.executeBatch(batchValues.toArray(new Map[accs.size()]));
}
.
public ArrayList<Account2> filterAccount(JdbcTemplate jdbcTemplate)
{
String sql= "query string";
ArrayList<Account2> searchresults = (ArrayList<Account2>) jdbcTemplate.query(sql,
new RowMapperResultSetExtractor<Account2>(new AccountRowMapper(), 130000));
return searchresults;
}
.
public class AccountRowMapper implements RowMapper<Account2> {
public Account2 mapRow(ResultSet rs, int rowNum) throws SQLException {
Account2 a = new Account2();
a.setId(rs.getString("id"));
a.setName(rs.getString("name"));
a.setOrganization__c(rs.getString("Organization__c"));
return a;
}
}
The question here is what is the quickest way for me to 'switch' over to using a SQL server to pull the data down, with the same table and rows, without changing too much of my current code?

stored procedure 'auto_pk_for_table' not found

I don't know why I received the error :
org.apache.cayenne.CayenneRuntimeException: [v.4.0.M5 Feb 24 2017 07:47:55] Commit Exception
[...]
Caused by: java.sql.SQLException: Procédure stockée 'auto_pk_for_table' introuvable.
[...]
I'm using Cayenne :
<dependency>
<groupId>org.apache.cayenne</groupId>
<artifactId>cayenne-server</artifactId>
<version>4.0.M5</version>
</dependency>
and JDTS for sql server :
<dependency>
<groupId>net.sourceforge.jtds</groupId>
<artifactId>jtds</artifactId>
<version>1.3.1</version>
</dependency>
The connexion is ok :
avr. 10, 2017 2:36:30 PM org.apache.cayenne.datasource.DriverDataSource getConnection
INFOS: +++ Connecting: SUCCESS.
I'm trying to create a new user (I'm starting by bascis!) so my code is :
(I cut a little bit, it's too long:!)
public abstract class _UserInfo extends CayenneDataObject {
public static final String ADDRESS_PROPERTY = "address";
public void setAddress(String address) {
writeProperty(ADDRESS_PROPERTY, address);
}
public String getAddress() {
return (String)readProperty(ADDRESS_PROPERTY);
}
}
public class UserInfo extends _UserInfo implements Serializable {
private static final long serialVersionUID = 1L;
public String address;
public String getAdress() {
return address;
}
public void setAddress(String address) {
super.setAddress(address);
}
//I have the hashcode and equals too
}
Then, I used vaadin to create my form :
public class UserAddView extends CustomComponent implements View {
private static final long serialVersionUID = 1L;
private TextField address;
private Button save;
public static final String USERVIEW = "user";
public boolean checkValidation() {
if (!checkTextFieldValid(address))
return false;
return true;
}
public boolean checkTextFieldValid(TextField element) {
if (element == null || element.isEmpty()) {
Notification.show(
"You should register a " + element.getDescription(),
Type.WARNING_MESSAGE);
return false;
}
return true;
}
public UserAddView() {
VerticalLayout mainLayout = new VerticalLayout();
mainLayout.setSizeFull();
setCompositionRoot(mainLayout);
final VerticalLayout vlayout = new VerticalLayout();
address = new TextField("Address:");
address.setDescription("Address");
vlayout.addComponent(address);
save = new Button("Save");
vlayout.addComponent(save);
mainLayout.addComponent(new HeaderMenu());
mainLayout.addComponent(vlayout);
addListeners();
}
private void addListeners() {
save.addClickListener(new ClickListener() {
private static final long serialVersionUID = 1L;
#Override
public void buttonClick(ClickEvent event) {
if (checkValidation() == true) {
ServerRuntime cayenneRuntime = ServerRuntime.builder()
.addConfig("cayenne-myapplication.xml").build();
ObjectContext context = cayenneRuntime.newContext();
UserInfo user = context.newObject(UserInfo.class);
user.setAddress(address.getValue());
user.getObjectContext().commitChanges();
Notification.show(
"Has been saved, We will send you your password by email. Your user login is: "
+ email.getValue(), Type.TRAY_NOTIFICATION);
getUI().getNavigator().navigateTo(HomepageView.MAINVIEW);
}
}
});
}
#Override
public void enter(ViewChangeEvent event) {
// TODO Auto-generated method stub
}
}
EDIT, add information : In my user object, I have a userid (primary key), in cayenne I wrote it as primary key too and in smallint. This error seems to be link... https://cayenne.apache.org/docs/3.1/api/org/apache/cayenne/dba/sybase/SybasePkGenerator.html
The error happens when you insert a new object. For each new object Cayenne needs to generate a value of the primary key. There are various strategies to do this. The default strategy depends on the DB that you are using. For SQLServer (and for Sybase, as you've discovered :)) that strategy is to use a special stored procedure.
To create this stored procedure (and other supporting DB objects), go to CayenneModeler, open your project, and select "Tools > Generate Database Schema". In "SQL Options" tab, uncheck all checkboxes except for "Create Primary Key Support". The SQL you will see in the window below the checkboxes is what you need to run on SQL server. Either do it from Cayenne modeler or copy/paste to your favorite DB management tool.
There's also an alternative that does not require a stored procedure - using DB auto-increment feature. For this you will need to go to each DbEntity in the Modeler and under the "Entity" tab select "Database-Generated" in the "Pk Generation Strategy" dropdown. This of course implies that your PK column is indeed an auto-increment in the DB (meaning you may need to adjust your DB schema accordingly).

How to persistent Map in JPA in GAE

I don't know why I can't persistent MAP in JPA in GAE
AnnualReport thatyear = .......
if (stud.getAnnualReport() == null){
Map<Integer,AnnualReport> temp = new HashMap<Integer,AnnualReport>();
temp.put(thatyear.getAttrKey(), thatyear);
stud.setAnnualReport(temp);
} else{
Map<Integer,AnnualReport> temp2 = stud.getAnnualReport();
temp2.put(thatyear.getAttrKey(), thatyear);
stud.setAnnualReport(temp2);
}
em.getTransaction().begin();
try {
em.persist(stud);
em.getTransaction().commit();
} finally {
if (em.getTransaction().isActive()) {
em.getTransaction().rollback();
}
}
Actually in http:// localhost :8888/_ah/admin/datastore I can see the thatyear has been persistent; However, I can never get them; or, stud.getAnnualReport() is always empty.
EntityManager em;
em = EMF.get().createEntityManager();
AnnualReport thatyear = stud.getAnnualReport().get(yearselected);
I really don't know what to do. Following is the relationship between Stud & AnnualReport
Stud
#Entity( name = "Stud")
public class Stud{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Key studID;
private String lastName = new String();
private Map<Integer,AnnualReport>annualReport = new HashMap<Integer,AnnualReport>(20);
#OneToMany(mappedBy="stud",cascade = CascadeType.ALL)
#MapKey(name = "attrKey")
#Basic
public Map<Integer, AnnualReport> getAnnualReport() {
return annualReport;
}
AnnualReport
#Entity( name = "AnnualReport")
public class AnnualReport implements Serializable{
private static final long serialVersionUID = 3581307841164176872L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Key annualReportID;
public int attrKey;
#ManyToOne
Stud stud;
private String attendSchoolNote;
I don't know what happens. Why I can't get those map information which are already persistent?
No idea why you don't get the expected result, but then you present no debug info. You can easily follow the persistence process using the log, telling you what is actually persisted into the GAE Entity objects. GAE has a (JDO) unit test at
http://code.google.com/p/datanucleus-appengine/source/browse/trunk/tests/com/google/appengine/datanucleus/jdo/JDOMapTest.java
which demonstrates correct behaviour (and since JDO/JPA is simply a wrapper over the persistence engine, no reason to think the same would not persist fine using JPA).
Edit : in fact I just added a test for JPA maps at http://code.google.com/p/datanucleus-appengine/source/browse/trunk/tests/com/google/appengine/datanucleus/jpa/JPAMapTest.java and works fine.

Resources